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ABSTRACT 


The  attack  on  the  USS  Cole  within  a  civilian  port,  and  the  increased  threat  of 
pirating  and  terrorism  on  the  high  seas,  underscore  the  immediate  need  for  a  maritime 
non-lethal  capability.  This  research  uses  modeling  and  simulation  to  explore  the 
requirements  and  tactical  use  of  non-lethal  capabilities  in  a  maritime  force  protection 
mission.  Specifically,  a  multi-agent  simulation  emulates  a  tactical-level  mission  in  which 
a  U.S.  Navy  vessel  returning  to  Naval  Station,  Norfolk,  VA,  encounters  a  variety  of 
maritime  surface  threats.  Data  farming  is  the  method  used  to  address  the  research 
questions  by  applying  high  performance  computing  to  the  simulation  model,  with  the 
intent  of  examining  a  wide  range  of  possibilities  and  outcomes.  The  non-lethal 
capabilities  are  analyzed  in  their  effectiveness  to  1)  determine  intent,  2)  deter  inbound 
surface  vessels,  and  3)  engage  targets  identified  as  hostile  through  the  continuum  of 
force. 
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EXECUTIVE  SUMMARY 


The  terrorist  attack  on  the  USS  Cole  on  October  10,  2000,  re-energized  national 
efforts  towards  preserving  freedom  of  the  seas  and  safe  access  to  ports,  with  a  directed 
focus  on  force  protection  initiatives  and  technology.  The  tremendous  potential  of  non- 
lethal  capabilities  in  maritime  force  protection  has  been  recognized  by  the  Quadrennial 
Defense  Review  as  well  as  an  independent  study  conducted  by  the  Naval  Studies  Board. 
This  research,  sponsored  by  the  Joint  Non-Lethal  Weapons  Directorate,  directly  impacts 
the  current  development  of  non-lethal  requirements  and  tactics  needed  for  effective 
maritime  security. 

To  scope  the  analysis  efforts,  the  following  research  questions  of  interest  were 
identified  for  an  entering  port  force  protection  mission: 

•  What  non-lethal  capabilities  are  required  for  a  maritime  force  protection 
mission? 

•  When  are  non-lethal  capabilities  tactically  appropriate? 

•  What  are  the  geographical  effects? 

•  How  are  non-lethal  capabilities  used  to  identify  threats  from  non-threats? 

•  Is  Multi- Agent  Simulation  (MAS)  an  appropriate  modeling  tool? 

The  first  step  in  answering  these  questions  was  to  develop  a  scenario  within  the 
MAS  environment  that  appropriately  emulated  a  Navy  ship’s  ability  to  do  three  tasks:  1) 
Identify  potential  threats;  2)  Determine  intent  of  approaching  small  vessels;  and  3)  Deter 
vessels  from  closing  within  the  100  yard  naval  protective  zone. 

With  the  scenario  developed,  the  next  step  was  to  apply  an  efficient  design  of 
experiment  (DOE)  using  distributed  simulation  in  order  to  capture  a  reasonable  region  of 
possible  outcomes.  The  final  scenario  was  simulated  over  a  range  of  33  input  variables. 
These  input  factors  included  everything  from  requirements  of  the  non-lethal  capabilities 
such  as  range  and  firing  rates,  to  quantified  intangibles,  such  as  fear  and  aggression  levels 
of  inbound  targets. 


XIX 


Through  the  power  of  simulation,  over  three  quarters  of  a  million  data  points  were 
generated.  The  key  element  in  tying  the  data  to  the  research  questions  was  the  Measures 
of  Effectiveness  (MOEs)  selected  for  analysis.  The  following  MOEs  were  defined 
specifically  for  this  research: 

•  MOE  1  -  Deterrence  Ratio:  The  percentage  of  time  the  targets  are 
deterred. 

•  MOE  2  -  Hostile  Identification  Ratio:  The  percentage  of  time  targets  are 
identified  as  hostile,  engaged,  and  subsequently  killed,  with  lethal  force. 

•  MOE  3  -  Warning  Zone  Identification  Ratio:  If  identified  as  hostile,  the 
percentage  of  time  the  targets  are  identified,  using  non-lethal  capabilities, 
outside  the  threat  zone. 

Using  multiple  data  mining  techniques,  ordinary  statistics,  and  visualization  tools, 
the  mammoth  amount  of  data  was  efficiently  analyzed  to  provide  insight  on  the  research 
questions.  Key  insights  into  non-lethal  requirements  and  tactical  application  include: 

•  The  employment  of  non-lethal  capabilities  is  extremely  effective  when 
used  to  identify  threats  from  non-threats  in  an  ambiguous  situation. 

•  Inbound  speed  is  the  critical  factor  in  identifying  and  engaging  inbound 
hostile  threats  outside  of  the  exclusion  zone. 

•  The  number  of  inbound  targets  has  little  to  no  impact  on  identification  and 
engagement  rates  of  hostile  targets. 

•  The  first  response  non-lethal  capability  is  the  most  crucial  in  deterring 
non-suicidal  targets. 

•  The  AHD  is  significantly  more  effective  when  employment  time  is  less 
than  30  seconds  against  hostile  targets  and  1  minute  for  neutral  or  loitering 
targets. 

•  When  used  alone,  counter-personnel  non-lethal  capabilities  fail  to  deter 
loitering  targets  who  attack  when  within  close  proximity. 

With  a  limited  number  of  non-lethal  capabilities  applied  to  a  very  specific 
mission,  this  thesis  researched  what  other  possible  applications  are  appropriate  using  the 
methodology  applied.  Follow-on  work  was  identified  for  three  primary  research  areas: 

Requirements  -  This  research  modeled  three  counter-personnel  non-lethal 
capabilities.  Future  work  should  include  scenario  expansion  to  include  counter-material 
capabilities. 


XX 


Tactics  -  The  scenario  simulated  involved  one  primary  tactic  adapted  from 
current  U.S.  Navy  tactics.  Future  work  should  include  a  comparative  analysis  of  new 
tactics,  especially  in  areas  where  this  research  deemed  current  tactics  fall  short. 

Vulnerability  Assessments  -  In  addition  to  exploring  the  requirements  and  tactics 
of  the  non-lethal  capabilities,  this  research  was  very  effective  in  exploring  the  geographic 
vulnerabilities  of  the  modeled  port.  Given  this  success,  future  work  should  apply  this 
methodology  to  other  geographic  ports  or  chokepoints  of  interest  in  order  to  assist  in  anti¬ 
terrorism/force  protection  planning  prior  to  ship  arrival. 

In  summary,  this  research  used  cutting  edge  modeling  and  simulation  to 
effectively  emulate  a  complex  scenario  where  little  historical  performance  data  exists. 
This  thesis  produced  valuable  insights  through  application  of  proven  operations  research 
tools  and  techniques,  and  provides  a  revolutionary  compliment  to  subject  matter  expertise 
in  non-lethal  requirements  and  tactics  development  early  in  the  acquisition  process. 
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I. 


INTRODUCTION 


Preserving  the  freedom  of  the  seas  is  a  top  national  priority.  The  right  of 
vessels  to  travel  freely  in  international  waters,  engage  in  innocent  and 
transit  passage,  and  have  access  to  ports  is  an  essential  element  of  national 
security.* 

The  National  Strategy  for  Maritime  Security 

A.  BACKGROUND  OF  NON-LETHAL  CAPABILITIES 
I.  Defining  the  Need  for  Non-Lethal  Capabilities 

Libya,  Lebanon,  Panama,  Somalia,  Bosnia,  Haiti,  and  Kosovo:  Over  the  past  20 
years,  these  are  the  countries  that  have  been  the  theater  of  operations  for  the  U.S. 
military.  Yet,  despite  the  presence  and  action  of  the  U.S.  military,  none  of  these 
operations  were  deemed  wars.  In  fact,  the  last  Congressional  declaration  of  war  was 
World  War  II  {Declaration  of  War  by  the  United  States,  2006),  yet  U.S.  military  power 
has  been  projected  many  times  since.  After  the  Cold  War,  the  U.S.  shifted  its  policies 
when  committing  its  forces.  Specifically,  U.S.  decision  makers  expanded  from  the  use  of 
military  force  strictly  for  conventional  war  to  the  application  of  forces  over  a  wide 
spectrum  of  military  operations.  This  shift  in  the  application  of  military  force  has  had  a 
significant  impact  on  training,  doctrine,  and  capabilities  of  U.S.  forces. 

This  is  most  clearly  exemplified  in  the  differing  national  strategies  between  the 
two  eras.  NSC-68,  which  was  the  guiding  principal  for  national  strategic  objectives  in 
the  Cold  War,  defines  four  possible  courses  of  action:  1)  Continuation  of  current 
policies;  2)  Isolation;  3)  War;  and  4)  A  more  rapid  building  up  of  the  political,  economic, 
and  military  strength  (National  Security  Council,  1950).  Of  these,  war  is  the  only  action 
which  incorporates  actual  use  of  military  forces. 

In  reaction  to  the  first  attack  on  the  World  Trade  Center,  the  two  attacks  on  U.S. 
embassies  in  Africa,  the  assault  on  the  USS  Cole,  and  finally,  the  mass  terrorist  attack  on 
the  Twin  Towers  and  the  Pentagon  in  September  2001,  the  administration  adopted  a  new 

*  Department  of  Defense  and  Homeland  Security.  (2005).  The  National  Strategy  for  Maritime  Security. 
Washington  D.C. 
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approach  towards  the  National  Security  Strategy  that  fundamentally  differed  from  any 
seen  before.  Specifically,  the  Bush  administration  embraced  preventive  war.  The  most 
significant  point  of  this  National  Security  Strategy  is  the  motivation  to  go  beyond 
preemptive  strikes  and  adopt  a  policy  towards  preventative  action  before  threats  even 
have  the  chance  to  develop. 


Given  the  U.S.  military’s  experience  in  non-wartime  situations,  military  doctrine 
encompasses  the  concepts  of  the  current  National  Security  Strategy.  The  range  of 
military  operations  (ROMO),  as  shown  in  Figure  1,  describes  military  operations 
extending  from  war  to  military  operations  other  than  war  (MOOTW). 


RANGE  OF  MILITARY  OPERATIONS 
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Counterdrug 
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Protection  of  Shipping 
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UO  wIVII 

Authorities 

US  Civil  Support 

Figure  1.  Range  of  Military  Operations  (From  Chairman  Joint  Chief  of  Staff,  1995) 


The  idea  of  preventative  action  is  captured  within  the  purpose  of  MOOTW,  which 
is  to  deter  war  and  promote  peace.  Because  MOOTW  are  preventative  in  nature  and 
generally  involve  non-combatant  civilians,  they  follow  more  restrictive  rules  of 
engagement  (ROE)  (Chairman  Joint  Chief  of  Staff,  1995).  As  such,  non-lethal 
capabilities  have  been  identified  as  a  significant  tool  in  accomplishing  MOOTW 
missions,  as  well  as  enhancing  mission  effectiveness  in  conventional  combat  operations. 
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2.  History  of  Non-Lethal  Capabilities  Development 

Although  numerous  examples  of  non-lethal  employment  ean  be  extrapolated  from 
history,  the  first  formal  instanee  of  non-lethal  capabilities  directly  utilized  to  support  a 
U.S.  military  objective  is  the  U.S.  intervention  in  Somalia.  LtGen  Anthony  Zinni,  tasked 
with  the  withdrawal  of  UN  troops  from  Mogadishu,  called  for  the  immediate  fielding  of 
non-lethal  capabilities  in  order  to  minimize  casualties  and  mitigate  escalation  {Non-lethal 
Weapons,  2005). 

By  1996,  the  Department  of  Defense  released  Directive  3000.3,  officially 
establishing  the  Joint  Non-Lethal  Weapons  Program  (JNLWP),  Non-Lethal  policies,  and 
DOD-wide  responsibilities  for  non-lethal  development.  From  this,  the  Commandant  of 
the  Marine  Corps,  serving  as  the  executive  agent  (EA)  for  JNLWP,  established  the  Joint 
Non-Lethal  Weapons  Directorate  (JNLWD).  The  JNLWD  serves  as  the  central  liaison 
for  all  issues  involving  non-lethal  capabilities,  including  research  and  development, 
science  and  technology,  requirements,  and  acquisition  of  non-lethal  technologies  {The 
Joint  Non-Lethal  Weapons  Program,  2005). 

In  December  2002,  the  Defense  Department’s  Joint  Requirements  Oversight 
Council  (JROC)  approved  a  Mission  Need  Statement  (MNS)  expressing  the  need  for  non- 
lethal  capabilities  in  order  to  “break  the  cycle  of  violence  that  often  prolongs  or  escalates 
conflict”  (Allison  et  al.,  2004).  In  2004,  the  Council  on  Foreign  Relations  released  a 
report  recommending  more  intensive  use  of  current  non-lethal  capabilities  as  well  as 
aggressive  development  of  new  non-lethal  concepts  (Allison  et  al.,  2004).  In  response, 
the  JNLWP  EA  issued  a  directive  requiring  a  Capabilities-Based  Assessment  (CBA)  for 
all  non-lethal  weapons  and  capabilities.  In  accordance  with  the  Chairman  of  the  Joint 
Chiefs  of  Staff  Instruction  3170.01  for  the  Joint  Capabilities  Integration  and 
Development  System  (JCIDS),  the  JNLWD  spearheaded  a  JCIDS  analysis.  The 
objectives  of  this  analysis  were  to  meet  the  LA’s  directive  and  update  the  previous  JROC 
MNS  {Functional  Area  Analysis,  2004).  Figure  2  depicts  the  objective  of  the  JCIDS 
analysis  at  each  step. 
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JCIDS  ANALYSIS 

(DoD  Prescribed  Four-Step  Process) 

Functional  Area  Analysis 

—  Devel  opOperationaiT  asks ,  Con  d  it  ior  s,  an  d  Stan  d  ard  s 
Functional  Needs  Analysis 

—  Determine  Gaps  and  Timeframe 
Functional  Solutions  Analysis 

—  DOTMLPF  Analysis 

—  Ideas  for  Materiel  Approaches 

—  An  a  lys  i  s  of  Materi  e  I  Ap  proac  ties 
Post  Independent  Analysis 

—  Sponsor  Prioritizes  Solutions  and  Actions 


Figure  2.  The  Four  Steps  in  the  JCIDS  Process  (From  Functional  Area  Analysis,  2004) 

3.  Non-Lethal  Capabilities  and  the  Maritime  Environment 

The  attack  on  the  USS  Cole  within  a  civilian  port,  and  the  increased  threat  of 
pirating  and  terrorism  on  the  high  seas,  underscored  the  immediate  need  for  a  maritime 
non-lethal  capability.  This  research  seeks  to  focus  on  non-lethal  capabilities  in  the 
maritime  environment,  and  therefore  will  only  address  elements  of  the  JCIDS  analysis 
which  focus  on  the  key  maritime  terrain.  The  JCIDS’  Functional  Area  Analysis  (FAA) 
determined  which  maritime  tasks  can  be  supported  by  non-lethal  capabilities.  The 
conclusion  was  the  identification  of  18  tasks  that  apply  to  the  maritime  terrain.  The  FAA 
validated  the  relevance  of  non-lethal  capabilities  in  accomplishing  critical  missions  at 
sea.  The  second  JCIDS  step  was  to  conduct  the  Functional  Needs  Analysis  (FNA),  one 
of  the  two  major  findings  identified  that  current  and  projected  non-lethal  capabilities  fail 
to  satisfy  conditions  involving  maritime  tasks.  Further  conclusions  showed  that 
performance  gaps  existed  in  most  standards  required  of  non-lethal  capabilities  to  include 
range,  coverage,  duration,  effects  and  reversibility  {Functional  Needs  Analysis,  2004). 
Based  on  the  gaps  identified  in  the  FNA,  the  Functional  Solutions  Analysis  (FSA) 
focused  on  material  and  non-material  solutions  needed  for  mitigation.  The  primary 
finding  of  the  FSA  was,  “Gaps  identified  during  the  FNA  are  mainly  caused  by  the 
physical  limitations  of  current  non-lethal  capabilities”  {Functional  Solutions  Analysis, 
2005). 
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In  a  separate  assessment,  the  Naval  Studies  Board  defined  the  need  for  non-lethal 
capabilities  in  a  variety  of  U.S.  Navy  missions.  The  assessment  divided  the  missions  into 
defensive  and  offensive  objectives.  The  primary  defensive  mission  identified  was  force 
protection  while  the  offensive  missions  included  maritime  interdiction,  blockades  and 
strikes  (Naval  Studies  Board,  2003).  In  its  conclusions,  the  assessment  noted  four  areas 
of  concern.  The  following  two  contribute  to  the  underlying  motivation  for  this  research: 

•  Without  a  much  stronger  overall  program  to  understand  and  characterize 
the  effects  and  effectiveness  of  non-lethal  weapons,  commanders  will 
remain  reluctant  to  request  or  employ  them  (Naval  Studies  Board,  2003). 

•  Without  concepts  for  the  use  of  non-lethal  weapons,  developers  will  not  be 
successful  in  focusing  ideas  and  programs  (Naval  Studies  Board,  2003). 

B.  OBJECTIVE 

The  JCIDS  FSA  conclusion  that  new  non-lethal  capabilities  are  needed  and  the 
Naval  Studies  Board  assessment  that  understanding  effectiveness  and  developed  concepts 
are  critical  for  non-lethal  implementation  drive  the  main  questions  investigated  in  this 
work.  This  research  seeks  to  use  modeling  and  simulation  to  explore  the  requirements, 
effectiveness  and  tactical  use  of  non-lethal  capabilities  for  a  maritime  force  protection 
mission. 

C.  SCOPE  OF  THESIS  AND  RESEARCH  QUESTIONS 

This  research  will  focus  on  a  tactical-level  scenario  in  which  a  U.S.  Navy  vessel 
returning  to  Naval  Station  (NAVSTA),  Norfolk,  VA,  will  encounter  a  variety  of  maritime 
surface  threats.  The  objective  of  the  U.S.  Navy  vessel  is  to  reach  the  terminal  point 
(NAVSTA  pier).  En  route  to  the  objective  point,  the  U.S.  Navy  vessel  will  employ  non- 
lethal  capabilities  as  appropriate. 

The  primary  research  question  of  this  thesis  is: 

•  What  non-lethal  capabilities  are  required  in  a  maritime  force  protection 
environment  in  order  to  effectively  determine  intent  and/or  deter 
suspicious  small  vessels? 

Supporting  questions  are  as  follows: 

•  Is  a  multi-agent  simulation  (MAS)  an  appropriate  means  to  represent  and 
analyze  a  ship’s  ability  to  determine  intent  and  use  continuum  of  force? 

•  In  what  region  of  conditions  are  non-lethal  capabilities  tactically 
appropriate  against  high  speed  hostile  threats? 
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•  What,  if  any,  are  the  major  differences  in  requirements  given  varying 
geographic  constraints? 

•  What  requirements  are  needed  of  non-lethal  capabilities  in  order  to 
effectively  separate  threats  from  non-threats  in  a  homogeneous  group? 

D.  METHODOLOGY 

The  systematic  approach  of  this  research  is  to  apply  a  process  called  data  farming, 
an  analytic  process  initially  designed  to  support  research  and  development  efforts  of  the 
Marine  Corps  Warfighting  Laboratory’s  Project  Albert.  Data  farming  is  a  method  used  to 
address  decision  makers’  questions  by  applying  high  performance  computing  to  a 
simulation  model,  with  the  intent  of  examining  a  wide  range  of  possibilities  and 
outcomes  {Project  Albert,  2006).  By  incorporating  cutting  edge  experimental  designs 
developed  at  the  Naval  Postgraduate  School  into  the  data  farming  process,  thousands  of 
data  points  are  collected  and  analyzed  in  order  to  identify  significant  factors,  interactions 
between  key  variables,  and  critical  performance  thresholds  (Cioppa,  2002;  Kleijnen  et  al., 
2005;  Cioppa  and  Lucas,  2006). 

The  following  chapters  provide  the  specific  background  on  each  facet  of  the  data 
farming  process  as  applied  to  non-lethal  capabilities  in  the  maritime  environment. 
Chapter  II  describes  the  tactical  scenario  designed  for  this  research.  The  chapter 
discusses  the  assumptions  and  methodology  used  to  emulate  tactically-significant 
interactions  between  maritime  vessels  using  a  multi-agent  simulation  (MAS).  Chapter  III 
addresses  how  a  robust  set  of  data  is  created  by  implementing  an  efficient  design  of 
experiment  (DOE)  and  leveraging  high  power  computing  power.  Details  include  which 
initial  conditions  and  parameters  will  be  varied  in  order  to  explore  the  wide  spectrum  of 
possible  real-world  outcomes.  Chapter  IV  focuses  on  the  analysis  of  the  data  after  the 
DOE  is  applied.  Specifically,  this  chapter  discusses  the  motivation  behind  the  measures 
of  effectiveness  (MOE)  selected,  how  these  MOEs  are  extracted  from  the  simulation 
process,  and  what  statistical  insight  is  garnered  in  answering  the  primary  research  and 
future  applications  of  this  research. 
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II.  SCENARIO  DESCRIPTION 


It  is  impossible  to  foresee,  or  to  fully  comprehend,  all  the  challenges  we 
will  face.  But  by  building  a  balanced  force  that  is  resilient  and  adaptable, 
with  the  depth  of  capabilities  required  to  meet  the  demands  of  a  multi¬ 
mission,  multi-task  environment,  we  can  mitigate  against  this  uncertainty.^ 

Admiral  Mike  Mullen 

A.  OVERVIEW 

The  primary  objective  of  this  research  is  to  use  modeling  and  simulation  to 
capture  the  required  non-lethal  capabilities  to  deter  terrorist  attack  or  determine  hostile 
intent  of  maritime  assets.  A  realistic  scenario  within  the  modeling  and  simulation 
environment  provides  the  foundation  for  this  analysis.  This  chapter  describes  the  model 
selection  and  scenario  design  implemented  for  this  specific  research. 

B.  MODEL  SELECTION 

I.  Multiple  Agent  Simulations 

Determination  of  intent  and  deterrence  of  a  target  are  both  decisions  made  by  one 
side  or  the  other.  In  the  real  world,  many  factors  such  as  weather,  terrain,  and  the 
physical  capabilities  and  limitations  of  the  players  drive  the  decision  making  process. 
These  features  can  be  captured  in  a  physics-based  or  probability-based  model.  However, 
human  factors,  such  as  morale,  confidence,  fatigue,  and  intelligence,  are  also  critical 
elements  of  the  hostile  decision  to  continue  the  attack  or  be  deterred.  They  are  also  key 
factors  in  the  shipboard  decision  to  recognize  the  threat  and  take  non-lethal  or  lethal 
action.  The  previously  mentioned  models  generally  ignore  these  factors.  For  this  reason, 
this  research  uses  multi-agent  simulation  (MAS)  in  order  to  capture  the  intangibles,  such 
as  fear  and  aggression,  which  are  essential  to  effectively  emulate  the  decision  making 
processes. 

Although  definitions  vary,  the  primary  distinction  of  MAS  is  that  it  is  made  up  of 
agents,  or  entities,  that  behave  autonomously  and  are  capable  of  choice  given  a  simple  set 
of  rules  or  logic  (Sanchez  and  Lucas,  2002).  Like  probability-based  models,  MAS 
incorporate  probabilities  in  order  to  account  for  random  effects.  One  goal  for  this 

t  Mike  G.  Mullen,  (2006).  CNO  Guidance  for  2006:  Meeting  the  Challenge  for  a  New  Era.  Retrieved 
July  8,  2006  from  http://www.globalsecurity.org/military/library/policy/navy/cno-guidance_2006.html. 
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scenario  is  to  simulate  the  choice  of  a  surface  target  to  deter.  This  choice  is  based  on 
how  influenced  the  target  is  by,  or  how  ‘afraid’  it  is  of,  the  U.S.  Navy  ship.  A  target’s 
decision  whether  or  not  to  deter  is  also  a  factor  of  how  aggressive  that  target  is.  For 
example,  a  target  that  is  hostile  in  nature  will  show  more  aggressiveness  and  less  fear 
than  a  neutral  target,  and  consequently  will  be  less  likely  to  deter.  These  levels  of  fear 
and  aggression  which  drive  agents’  choices  and  resultant  actions  are  captured  using 
MAS. 

2.  Why  PYTHAGORAS? 

The  MAS  chosen  for  this  research  is  PYTHAGORAS,  a  Northrop  Grumman 
product.  Three  reasons  influence  the  selection  of  this  model.  First,  PYTHAGORAS 
offers  a  unique  set  of  capabilities  appropriate  for  this  research  (Bitinas  et  al.,  2006). 
Specifically,  PYTHAGORAS 

•  Incorporates  soft  rules  to  distinguish  unique  agents; 

•  Uses  desires  to  motivate  agents  into  moving  and  shooting; 

•  Includes  the  concept  of  affiliation  (established  by  sidedness,  or  color 

value)  to  differentiate  agents  into  members  of  a  unit,  friendly  agents, 
neutrals,  or  enemies; 

•  Allows  for  behavior-changing  events  and  actions  (called  triggers)  that 
may  be  invoked  in  response  to  simulation  activities;  and 

•  Retains  traditional  weapons,  sensors,  and  terrain. 

Second,  PYTHAGORAS  is  specifically  designed  to  be  compatible  with  data 
farming,  the  primary  method  used  for  data  collection  and  analysis  in  this  research  (see 
Chapter  IV). 

Third,  as  a  U.S.  developed  model,  this  research  can  be  applied  to  quickly  assist 
our  operating  forces  in  exploiting  vulnerabilities  and  determining  tactics  to  mitigate  risk 
within  ports  and  choke  points  throughout  the  world. 

C.  MODEL  CONFIGURATION 

1.  Scale 

The  first  step  in  building  a  scenario  in  PYTHAGORAS  is  defining  what  a  time 
step  and  pixel  distance  mean.  All  subsequent  terrain  features,  weapons  and  sensor 
ranges,  firing  rates,  and  speeds  depend  on  the  pixel  distance  and/or  the  amount  of  time 
encompassed  by  a  time  step  (Bitinas  et  al.,  2006). 
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The  original  intent  was  to  create  a  scenario  representing  the  entrance  to  Thimble 
Shoals  Channel,  Norfolk,  VA,  to  the  Elizabeth  River,  Norfolk,  VA.  This  area  covers  a  25 
nautical  mile  by  15  nautical  mile  area.  When  translated  to  the  1000  pixel  by  1000  pixel 
PYTHAGORAS  terrain  box,  each  pixel  represented  nearly  50  yards.  In  a  real-world 
scenario  involving  a  high  speed  small  boat,  much  can  occur  over  50  yards.  In  order  to 
capture  more  range  fidelity,  the  real-world  area  of  interest  was  refined. 

The  scenario  was  re-scaled  to  a  9.5  nautical  mile  by  9.5  nautical  mile  area, 
focusing  on  the  entrance  to  the  Elizabeth  River  from  Thimble  Shoals  Channel. 
Translating  onto  the  1000  pixel  by  1000  pixel  terrain  box,  each  pixel  represents 
approximately  19  yards.  This  offers  almost  three  times  as  much  range  fidelity.  With  a 
time  step  defined  as  0.25  minutes,  the  blue  ship  takes  approximately  85  minutes  (300 
time  steps)  to  reach  its  objective  point.  Table  1  lists  how  various  units  of  distance,  time 
and  speed  are  represented  by  the  defined  pixel  and  time  step  measurements. 


Distance 

Pixel 

1 

10 

100 

Miles 

0.01 

0.11 

1.10 

Yards 

19.36 

193.60 

1936.00 

Kilometers 

0.02 

0.18 

1.77 

Meters 

17.70 

177.03 

1770.28 

Nautieal  Miles 

0.01 

0.10 

0.96 

Time 

Time  Step 

1 

4 

400 

Minutes 

0.25 

1.00 

100.00 

Hours 

0.00 

0.02 

1.67 

Speed 

Pixels/Time  Step 

1 

5 

11 

miles/hour 

2.64 

13.20 

29.04 

kilometers/hour 

4.25 

21.24 

46.74 

knots 

2.29 

11.47 

25.24 

Table  1.  Pixel  and  Time  Step  Translations 

2.  Terrain 

PYTHAGORAS  terrain  features  offer  many  options  to  include  height,  protection, 
concealment,  and  trafficability.  For  this  scenario,  grey  terrain  represents  land.  No  agents 
can  move  through  land  and  neither  detections  nor  engagements  can  occur  through  land. 
Yellow  terrain  represents  land  features  which  an  agent  cannot  pass  through,  but  do  not 
offer  100%  concealment  as  grey  terrain  does.  Blue  terrain  represents  water.  It  is 
assumed  that  the  blue  ship  will  remain  on  its  track  within  the  channel  and  all  small  boat 
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threats  have  sufficiently  small  drafts  to  traverse  across  water  of  any  depth.  Therefore,  the 
blue  terrain  is  100%  trafficable  and  all  targets  are  detectable  when  within  sensor  range. 
Figure  3  compares  the  actual  area  of  interest  versus  the  terrain  constructed  in  the 


PYTHAGORAS  model. 


Figure  3.  Comparison  of  Actual  Terrain  (From  Graphic  Retrieved  from  Google  Maps, 

2006)  and  PYTHAGORAS  Model  Terrain 

D.  BLUE  AGENTS 

In  PYTHAGORAS,  an  agent  class  refers  to  a  group  of  agents  that  share  the  same 
desires  and  capabilities.  Although  PYTHAGORAS  allows  for  complex  maneuvering,  all 
blue  (U.S.)  agents  are  modeled  to  move  together,  consistent  with  current  naval  policies 
and  procedures,  at  a  constant  speed  along  a  point-to-point  Plan  of  Intended  Movement 
(PIM).  The  purpose  for  this  maneuvering  scheme  is  twofold.  First,  although  observing  a 
target’s  reaction  to  own-ship  maneuvers  is  an  effective  means  in  determining  intent,  it  is 
not  always  possible.  In  an  entering  port  scenario,  it  is  realistic  that  the  entering  ship  is  in 
a  restricted  channel  operating  under  speed  restrictions.  In  this  situation,  a  U.S.  Navy 
vessel  must  rely  on  non-lethal  capabilities  and  communications  to  determine  intent.  The 
second  reason  is  to  prevent  so  much  complexity  in  the  model  that  the  effects  of  the  non- 
lethal  capabilities  cannot  be  analyzed  effectively.  Figure  4  highlights  the  PIM  of  the  blue 
ship  throughout  the  scenario. 


10 


Figure  4.  Blue  Ship  PIM 


For  this  scenario,  all  blue  agents  represent  a  single  U.S.  Navy  ship.  Each  blue 
agent  represents  a  capability,  either  non-lethal  or  lethal.  The  motivation  for  creating 
separate  agent  classes  for  each  capability  is  to  capture  the  rules  of  engagement  (ROE)  in 
the  scenario.  In  other  words,  by  defining  different  agent  classes,  the  blue  ship  can  assign 
the  appropriate  capability  to  different  types  of  targets.  For  example,  a  target  that  is 
recently  detected  and  has  not  been  queried  is  eligible  to  be  engaged  by  the  Acoustic 
Hailing  Device,  but  not  by  a  lethal  weapon.  The  five  blue  agent  classes  are:  1)  Acoustic 
Hailing  Device  (AHD),  2)  Optical  Dazzler  (OD),  3)  Warning  Munitions  (WM),  4)  Lethal, 
and  5)  Lethal  Exclusion  Zone  (EZ). 

1.  Non-Lethal  Capabilities 

a.  Acoustic  Hailing  Device 

AHDs  use  directed  acoustic  energy  to  provide  a  non-lethal  warning 
capability  by  producing  highly  directional  sound  beams.  These  sound  beams  can  be 
formed  into  either  warning  tones  or  voice  commands  {AHD  Fact  Sheet,  2006).  In 
November,  2005,  the  cruise  ship  “Seaborne  Spirit”  used  an  AHD  while  successfully 
repelling  armed  pirates  off  the  coast  of  Somalia  {Long  Range  Acoustic  Device,  2006). 
This  operational  use  in  the  maritime  environment  inspires  further  research  on  the 
effectiveness  of  this  capability  in  the  key  terrain  of  sea.  It  is  modeled  in  this  scenario  as  a 
direct  capability  with  varying  firing  rates  (depending  on  if  its  use  is  for  a  warning  tone  or 
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voice  commands).  Figure  5  shows  the  Long  Range  Acoustic  Device  (LRAD),  an  AHD 
developed  by  American  Technology  Corporation,  in  use  onboard  a  U.S.  Navy  patrol 
craft. 


Figure  5.  The  LRAD  onboard  USS  Typhoon  (From  Long  Range  Acoustic  Device, 

2006) 

b.  Optical  Dazzler 

An  OD  is  a  directed-energy  capability  that  employs  intense  visible  light, 
usually  generated  by  a  laser  {Dazzler,  2006).  The  purpose  of  an  optical  dazzler  is  to 
provide  visual  warning  and,  as  a  target  closes,  optical  disorientation.  Where  the  AHD 
provides  the  blue  ship  with  an  audio  capability,  the  OD  is  included  to  provide  the  blue 
ship  with  a  visual  capability.  Since  a  dazzler  are  typically  steady  or  strobe  light,  it  is 
modeled  in  this  scenario  as  a  direct  capability  with  a  high  rate  of  fire. 

c.  Warning  Munitions 

WM’s  are  projectile  cartridges  that  create  a  bright  flash,  loud  bang,  and 
smoke  when  fired  down  range  {JNLWM  Fact  Sheet,  2006).  Although  audio  and  visual 
capabilities  are  already  assigned  to  the  blue  ship,  the  WMs  model  a  more  aggressive 
warning  posture.  They  are  modeled  in  this  scenario  as  an  indirect  capability  where  a 
circular  error  probability  (CEP)  radius  originates  from  the  aim  point.  All  targets  within 
the  CEP  radius  are  affected  by  the  WMs  accordingly. 

2.  Lethal  Capabilities 

a.  Lethal 

Because  this  research  focuses  on  the  determination  of  intent  and  the 
escalation  of  force  up  to  lethal  means,  the  scenario  does  not  attempt  to  model  the  lethality 
of  a  specific  weapon  type.  Once  a  threat  has  been  identified  as  a  hostile  threat,  it  is 

engaged  with  a  ‘one  shot,  one  kill’  lethal  weapon.  Although  realistically  a  U.S.  Navy 
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ship  does  not  shoot  and  kill  every  target  identified  as  hostile,  this  feature  of  the  model  is 
used  later  for  data  analysis.  See  Chapter  4  for  a  more  detailed  discussion. 
b.  Lethal  Exclusion  Zone 

Navy  tactics  dictate  that  a  target  can  be  assumed  to  be  hostile  if  it  ignores 
all  warnings  and  continues  to  close  within  a  certain  distance.  To  capture  this  within  the 
model,  a  second  lethal  weapon  is  defined.  The  lethal  (EZ)  weapon  is  fired  when  any 
contact  closes  within  100  yards  of  the  blue  ship.  This  distance  derives  from  the  U.S. 
Coast  Guard  regulation  on  naval  vessel  protective  zones  stating  that  no  vessel  is  allowed 
within  100  yards  of  a  U.S.  naval  vessel  without  authorization  {Naval  Protective  Zones, 
2006).  The  purpose  in  having  two  lethal  weapons  defined  is  to  extract  which  contacts  are 
deemed  hostile  due  to  proximity  versus  those  identified  as  hostile  independent  of  range. 

E.  MODELING  NON-LETHAL  CAPABILITIES 

I.  Non-Lethal  Capabilities  Employment  Tactics 

The  tactics  used  by  the  blue  ship  to  engage  contacts  with  non-lethal  capabilities 
reflect  the  current  U.S.  Navy  force  protection  architecture.  Specifically,  three  zones  are 
created  around  the  blue  ship  for  a  layered  defense  (Naval  Studies  Board,  2003).  In  the 
first  zone,  the  assessment  zone,  the  blue  ship  must  detect  and  assess  if  a  target  is  a 
potential  threat.  At  scenario  start,  the  blue  ship  identifies  all  targets  as  neutral.  The  AHD 
is  used  to  make  an  initial  query  and  warn  targets.  In  the  second  zone,  the  warning  zone, 
the  blue  ship  uses  some  or  all  of  its  non-lethal  capabilities  to  deter  continuing  threats.  If  a 
target  reaches  the  third  zone,  the  threat  zone,  the  intent  of  the  approaching  craft  is 
assumed  to  be  hostile  (Naval  Studies  Board,  2003)  and  it  is  engaged  with  the  lethal  (EZ) 
weapon.  Figure  6  illustrates  the  force  protection  architecture  for  this  scenario. 
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■  AHD 

AHD  ODA\TM 

■  Letluil 

I  Letluil  (£Z) 


Figure  6.  Layered  Defense  against  Small  Boats  (After  Chief  of  Naval  Operations, 

2003) 

To  quote  the  Department  of  Defense  (DOD)  Directive  3000.3  “Policy  for  Non- 

Lethal  Weapons,”  released  on  July  9,  1996,  and  recertified  on  November  21,  2003: 

Neither  the  presence  nor  the  potential  effect  of  non-lethal  weapons  shall 
constitute  an  obligation  for  their  employment  or  a  higher  standard  for 
employment  of  force  than  provided  for  by  applicable  law.  In  all  cases,  the 
United  States  retains  the  option  for  immediate  use  of  lethal  weapons,  when 
appropriate,  consistent  with  international  law  (Department  of  Defense, 

2003). 

The  scenario  captures  this  directive  with  the  use  of  the  lethal  weapon.  Regardless 
of  range  or  what  non-lethal  capabilities  have  or  have  not  been  used,  the  blue  ship  will 
engage  with  lethal  force  in  the  first  or  second  zone  once  the  determination  of  hostile 
intent  has  been  made.  Appendix  A  demonstrates  in  detail  how  the  PYTHAGORAS 
sidedness  properties  are  used  to  model  the  continuum  of  force. 

2.  Non-Lethal  Capabilities  Effects 

When  talking  about  effects,  this  scenario  does  not  try  to  capture  the  physical 
reaction  of  the  non-lethal  capabilities  on  a  person  or  piece  of  equipment.  The  effects  in 
this  scenario  refer  to  the  targeting  and  maneuvering  decisions  of  the  agents  in  reaction  to 
the  employment  of  non-lethal  capabilities.  The  following  paragraphs  describe  the  overall 
concept  of  these  effects  and  how  they  control  the  blue  ship’s  determination  of  hostile 
intent  and  a  target’s  decision  to  deter. 
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a.  Suspicion  Level 

The  suspicion  level  represents  how  threatening  a  target  is  to  the  blue  ship. 
Each  time  a  non-lethal  capability  is  employed  against  a  target,  and  that  target  does  not 
react  or  deter,  the  suspicion  level  against  that  target  is  raised. 

b.  Suspicion  Level  Threshold 

The  suspicion  level  threshold  is  the  amount  of  suspicion  needed  against  a 
target  for  the  blue  ship  to  take  further  action.  The  most  significant  threshold  is  when 
enough  suspicion  has  been  built  to  declare  hostile  intent  and  escalate  to  lethal  force.  Sub- 
levels  represent  the  continuum  of  force  where  the  blue  ship  is  triggered  to  use  more 
aggressive  capabilities. 

c.  Deterrence  Level 

The  deterrence  level  represents  how  influenced  a  target  is  when  engaged 
by  non-lethal  capabilities.  This  level  is  cumulatively  collected  for  every  type  and 
instance  of  non-lethal  capability  employment  against  a  target. 

d.  Deterrence  Level  Threshold 

The  deterrence  level  threshold  is  the  amount  of  cumulative  influence 
needed  for  a  target  to  deter.  For  all  threat  types,  once  deterred,  a  target  will  maneuver 
away  from  the  blue  ship,  regardless  of  initial  movement  desires. 

3.  Hierarchy  of  Threats 

In  the  real  world,  a  U.S.  Navy  ship  is  not  equally  suspicious  of  all  inbound 
targets.  Watchstanders  use  the  kinematics  of  a  particular  threat  in  determining  its 
suspicion  level.  Two  important  kinematics  used  to  determine  intent  are  the  number  of 
targets  grouped  together  and  the  speed  of  those  targets.  To  capture  this  within  the 
scenario,  the  blue  ship  groups  inbound  threats  into  one  of  four  categories.  Category  I 
threats  incite  higher  levels  of  suspicion  than  Category  IV  threats  when  blue  ship  warning 
and  deterrence  efforts  are  unheeded.  Figure  7  defines  the  criteria  for  each  threat  category 
based  on  target  numbers  and  speeds. 
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•  Category  I:  3  or  more  targets  AND 
faster  than  18  knots 


♦  Category  II:  3  or  more  targets  OR 
faster  than  18  knots 


•  Category  III:  2  or  less  targets  AND 
slow  er  than  18  knots 

♦  Category  I^':  Fishing  or  loitering 
targets 

F igure  7 .  Hierarchy  of  Threats 
F.  THREAT  AGENTS 

The  purpose  of  the  threat  agents  in  this  scenario  is  to  provide  the  blue  ship  with  a 
realistic  selection  of  threats  that  a  U.S.  Navy  vessel  could  face  when  entering  a  port.  It  is 
important  to  understand  that  the  naming  convention  used  for  these  threats  (hostile, 
neutral,  and  suspect)  refers  to  the  target’s  true  identity  (ground  truth)  and  is  completely 
independent  of  the  blue  ship’s  classification  of  the  target. 

1.  Hostile  Targets 

Hostile  targets  are  designed  to  maneuver  on  a  direct  attack  run  towards  the  blue 
ship.  In  other  words,  they  act  as  “suicide  boats.”  As  such,  they  have  a  higher  deterrence 
threshold,  meaning  that  they  are  less  likely  to  be  deterred  by  non-lethal  capabilities. 
However,  once  the  deterrence  threshold  is  reached,  hostile  targets  will  break  off  the 
attack  run  for  a  variable  amount  of  time,  and  then  recommence  the  attack  run.  When  a 
hostile  target  re-commits  to  an  attack  run,  the  blue  ship  retains  all  prior  suspicion  levels 
built  up  against  that  target.  Conversely,  the  hostile  target’s  deterrence  level  is  re¬ 
initialized  below  the  deterrence  level  threshold. 

There  are  two  groups  of  hostile  targets  in  the  scenario  which  represent  two 
different  attack  tactics.  The  first  group  conducts  an  open  water  approach,  giving  the  blue 
force  early  target  detection  and  a  longer  prosecution  time.  The  second  group  conducts  a 
more  covert  maneuvering  scheme.  Here  the  enemy  hides  behind  a  jetty,  denying  early 
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detection.  When  the  blue  ship  nears  its  closest  point  of  approach,  the  second  group 
abandons  its  hiding  position  and  commences  its  attack  run.  Figure  8  illustrates  both 
hostile  maneuvering  tactics. 


JJJL 
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2.  Neutral  Targets 

Neutral  targets,  like  the  blue  ship,  have  their  own  PIM.  However,  since  the 
neutral  targets  are  closing  in  on  the  blue  ship  when  detected  and  will  pass  within  close 
proximity,  the  blue  ship  must  decide  if  the  approaching  ship  has  ill  intent  or  is  simply  a 
passerby.  Neutral  boats  have  a  lower  deterrence  threshold  and  are  more  likely  to  be 
deterred  by  non-lethal  capabilities.  After  the  deterrence  threshold  is  reached,  a  neutral 
target  will  still  proceed  to  its  next  waypoint,  but  will  maneuver  as  necessary  to  avoid  the 
blue  ship. 

There  are  two  types  of  neutral  targets  in  the  scenario  which  represent  two 
different  geographic  situations  when  the  blue  ship  reacts  to  the  closing  target.  The  first 
type  represents  a  channel  approach.  Although  the  blue  ship  has  early  target  detection, 
this  situation  forces  the  blue  ship  to  deal  with  multiple  types  of  threats  at  once  when 
transiting  through  the  chokepoint.  The  second  type  of  neutral  target  represents  a  crossing 
ferry.  Although  neutral  targets  are  more  easily  deterred,  this  situation  poses  a  geographic 
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constraint  where  the  neutral  target  may  not  have  room  to  clearly  maneuver  away  from  the 
blue  ship  even  though  its  deterrence  threshold  is  met.  Figure  9  illustrates  both  neutral 
maneuvering  schemes. 


Figure  9.  Neutral  Targets’  Movement 

3.  Neutral  Fishing/Loitering  Targets 

These  targets,  like  those  described  in  the  previous  section,  have  a  lower 
deterrence  threshold.  Yet  instead  of  following  a  PIM,  the  neutral  fishers  loiter  around  a 
chokepoint  through  which  the  blue  ship  must  transit.  Additionally,  they  respond  as  a 
group  in  that  once  one  of  the  targets  reaches  its  deterrence  threshold,  the  entire  group  will 
move  to  avoid  the  blue  ship. 

4.  Suspect  Fishing/Loitering  Targets 

The  suspect  fishers  represent  a  class  of  targets  that  are  not  as  aggressive  or  overt 
as  the  hostile  targets,  but  still  desire  to  attack  the  blue  ship.  At  scenario  start,  they  loiter 
with  the  neutral  fishers  at  a  chokepoint  and  await  a  “target  of  opportunity.”  This  creates 
the  appearance  of  a  homogenous  group  that  is  actually  composed  of  different  classes  of 
threats.  Once  the  blue  ship  moves  within  a  close  range,  a  suspect  fisher  will  change  its 
scheme  of  maneuver  and  commence  a  direct  attack  run  on  the  blue  ship.  Unlike  the 
hostile  targets,  once  a  suspect  fisher  reaches  its  deterrence  threshold,  it  will  move  as 
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necessary  to  avoid  the  blue  ship  (and  once  it  is  deterred,  it  will  remain  deterred).  Figure 
10  below  highlights  the  chokepoint  area  where  both  neutral  and  suspect  fishers  are 
positioned. 


Figure  10.  Chokepoint  with  Loitering  Targets 
G.  SUMMARY 

Although  this  scenario  represents  a  range  of  threats,  a  single  run  represents  only 
one  possible  outcome.  Since  little  or  no  data  exists  on  the  reaction  of  surface  maritime 
threats  to  newly  developed  non-lethal  capabilities  and  accurately  predicting  the  human 
factor  inputs  is  problematic,  a  single  run  is  not  sufficient  for  this  research.  The  next 
chapter  discusses  how  an  efficient  design  of  experiment  (DOE)  and  high  power 
computing  capabilities  are  used  to  analyze  the  trends  in  output  given  a  wide  variety  of 
possible  reactions  from  a  target. 
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III.  ANALYSIS  METHODOLOGY 


Any  intelligent  fool  can  make  things  bigger,  more  complex,  and  more 
violent.  It  takes  a  touch  of  genius,  and  a  lot  of  courage,  to  move  in  the 
opposite  direction.^ 

Albert  Einstein 

A.  OVERVIEW 

This  thesis  does  not  explore  the  physiological  effects  of  non-lethal  capabilities  on 
personnel.  The  ‘effectiveness’  in  this  scenario  relates  to  the  impact  non-lethal 
capabilities  have  on  the  tactical  objectives  to  1)  determine  intent,  2)  deter  inbound  surface 
vessels,  and  3)  engage  targets  identified  as  hostile  with  lethal  force. 

Figure  1 1  represents  modeling  and  simulation  as  a  function  where  the  output  of 
the  simulation  measures  the  effects  of  the  inputs.  Within  the  simulation  environment,  a 
set  of  input  parameters  are  applied  to  the  model  and  the  output  is  recorded.  The  input 
parameters  are  then  varied,  and  the  model  is  run  again.  This  iterative  process  is  repeated 
until  every  input  set  of  interest  is  applied  to  the  model.  The  resultant  Y  vector  is 
composed  of  literally  thousands  of  values,  representing  the  associated  output  for  each  set 
of  input  parameters.  The  methodology  of  this  thesis  is  to  build  the  right-side  of  the 
equation  so  that  the  resultant  output  is  pertinent  in  answering  the  initial  research 
questions. 

Model 

Y  =  F(X)  +  s  — 

Oulpuf  Inputs  Model 
Parameters 

Figure  11.  A  Model/Simulation  as  a  Big  Function  (From  OA4655,  2005) 

B.  MEASURES  OF  EFFECTIVENESS 

To  accomplish  this,  the  first  step  is  to  determine  what  output  is  needed  for  useful 
analysis.  One  type  of  output  is  a  measure  of  effectiveness  (MOE).  A  MOE  defined  is  a 
quantitative  measure,  generated  by  the  model,  used  to  compare  the  effectiveness  of 

t  Albert  Einstein.-  Albert  Einstein  Quotes.  Retrieved  July  21,  2006  from 
http://www.quotationspage.com/quote/2492 1  .html. 
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alternatives  in  achieving  the  tactical  objectives  (OA4655,  2005).  The  following  lists  the 
MOEs  designated  for  this  research  and  to  be  applied  in  follow-on  analysis. 

•  MOE  1  -  Deterrence  Ratio:  The  percentage  of  time  the  targets  are 
deterred. 

•  MOE  2  -  Hostile  Identification  Ratio:  The  percentage  of  time  targets  are 
identified  as  hostile,  engaged,  and  subsequently  killed,  with  lethal  force. 
This  MOE  not  only  captures  accurate  identification  of  hostile  threats,  but 
misidentification  of  neutral  targets. 

•  MOE  3  -  Warning  Zone  Identification  Ratio:  If  identified  as  hostile,  the 
percentage  of  time  the  targets  are  identified,  using  non-lethal  capabilities, 
outside  the  threat  zone. 

C.  VARIABLES  OF  INTEREST 

With  the  PYTHAGORAS  model  selected  and  the  scenario  constructed,  the  next 
step  is  to  define  which  input  variables,  or  factors,  are  of  interest.  Table  2  outlines  the 
factors  chosen  for  the  experimental  design.  Each  factor  is  varied  over  a  range  in  order  to 
evaluate  the  underlying  effects  on  the  output.  Each  factor  type  is  described  in  detail 
below. 
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Factor  # 

Factor  ID 

Factor  Name 

Min 

Pythagoras 

Value 

Max  Pythagoras 
Value 

1 

AHDMaxRange 

AHD  Maximum  Range 

25  (484  yds) 

80  (1580  yds) 

ODMaxRange 

OD  Maximum  Range 

15  (290  yds) 

55  (1065  yds) 

WMMaxRange 

WM  Maximum  Range 

5  (96  yds) 

25  (484  yds) 

4 

AHDFirRate 

AHD  Firing  Rate 

0.17  (1.5  min) 

0.5  (0.5  min) 

ODRedRadRat 

OD  Red  Radius  Ratio 

0.1 

0.9 

WMRedRadRat 

WM  Red  Radius  Ratio 

0.1 

0.9 

LethalRedRad 

Lethal  Red  Radius 

15 

50 

>=3  &>=18 

Color  Vulnerability  (Category  I) 

0.9 

1 

>=3  1  >=18 

Color  Vulnerability  (Category  II) 

0.7 

0.9 

10 

<3  &<18 

Color  Vulnerability  (Category  III) 

0.5 

0.7 

11-12 

TNFColVul;  TSFColVul 

Target  Fisher  Color  Vulnerability 
(Category  IV) 

0.1 

0.5 

13-14 

THAttVul;  TH2AttVul 

Target  Hostile  Attribute 
Vulnerability 

0.1 

0.6 

14-16 

TNAttVul;  TN2AttVul; 
TNFAttVul 

Target  Neutral  Attribute 
Vulnerability 

0.7 

1 

TSFAttVul 

Target  Suspeet  Attribute 
Vulnerability 

0.4 

0.8 

THInst;  TH2Inst 

Target  Hostile  Instanee 

1 

5 

21 

TNFInst 

Target  Neutral  Fisher  Instanee 

3 

7 

22 

TSFInst 

Target  Suspeet  Fisher  Instanee 

1 

4 

23-26 

THSpd;  TH2Spd; 
TNSpd;  TN2Spd 

Target  'Class'  Speed 

4  (9.2  kts) 

11  (25.2  kts) 

27 

THAlphaLess 

Target  Hostile  Alpha 

Less  Than  Value 

200 

240 

28 

TNAlphaLess 

Target  Neutral  Alpha 

Less  Than  Value 

230 

250 

29 

TSFAlphaLess 

Target  Suspeet  Fisher  Alpha 

Less  Than  Value 

215 

245 

Table  2.  Farmable  Factors 


1.  Blue  Factors 

a,  AHD  Maximum  Range 

This  is  the  maximum  range  of  the  AHD.  Although  different  AHDs  exist 
and  their  capabilities  are  known,  looking  at  variable  maximum  ranges  in  this  scenario 
serves  two  purposes.  First,  currently  fielded  AHDs  are  mainly  intended  for  land  use. 
Varying  the  range  of  the  AHD  in  this  scenario  explores  the  requirements  needed  over 
water.  Second,  this  factor  captures  the  performance  of  the  AHD  in  varying  conditions. 
For  example,  if  the  weather  is  very  windy,  thereby  reducing  the  maximum  effective  range 
of  the  AHD,  the  data  are  still  relevant  for  analysis. 

b.  OD  Maximum  Range 

This  is  the  maximum  range  of  the  OD.  Like  the  AHD,  this  variable 

explores  requirements  and  effectiveness  given  varying  conditions.  In  this  case,  different 

states  of  visibility  affect  the  maximum  range  of  the  OD. 
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c.  WM  Maximum  Range 

This  is  the  maximum  range  of  the  WM.  As  before,  this  variable  explores 

requirements  and  effeetiveness  given  varying  eonditions. 

d.  AHD  Firing  Rate 

This  defines  the  suppression  duration  of  the  AHD.  Since  its  potential  use 
ranges  from  immediate  audio  warnings  to  deliverable  voice  messages,  this  variable 
represents  the  time  needed  to  employ  the  AHD  against  a  target.  For  example,  a  longer 
firing  rate  represents  the  total  time  needed  to  broadcast  a  verbal  message. 

e.  Color  Vulnerability 

The  blue  ship  discriminates  against  inbound  targets  based  on  target 
kinematics,  as  discussed  in  Chapter  2  (Section  E.3).  This  factor  defines  the  suspicion 
level  assigned  to  each  category  in  the  hierarchy  of  threats.  Category  I  threats  (target 
groups  with  three  or  more  vessels  traveling  faster  than  1 8  knots)  create  the  highest  level 
of  suspicion,  and  in  PYTHAGORAS  terms,  are  assigned  the  highest  color  vulnerability. 

f.  OD  Red  Radius  Ratio 

This  defines  the  suspicion  level  needed  against  a  target  before  it  is  eligible 
to  be  engaged  by  the  OD.  This  level  will  always  be  a  proportion  of  the  WM  red  radius. 

g.  WM  Red  Radius  Ratio 

This  defines  the  suspicion  level  needed  against  a  target  before  it  is  eligible 
to  be  engaged  by  the  WM.  This  level  will  always  be  a  proportion  of  the  Lethal  red 
radius. 

h.  Lethal  Red  Radius 

This  represents  the  suspicion  level  threshold.  Once  this  level  is  reached,  a 
target  is  identified  as  hostile  and  is  immediately  engaged  by  lethal  force. 

2.  Target  Factors 

a.  Target  Instance 

This  is  the  number  of  targets  grouped  together  for  a  particular  threat  type 
and  location.  This  factor  applies  to  the  two  hostile  threat  types  and  both  neutral  and 
suspect  loitering  types.  For  the  neutral  targets  which  represent  an  outbound  vessel  in  the 
channel  and  a  ferry  crossing  the  channel,  these  threat  categories  are  fixed  at  only  one 
agent  through  all  iterations. 
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b.  Target  Speed 

This  is  the  average  speed  of  the  targets  within  a  threat  class.  This  factor 
applies  to  all  non-loitering  threats. 

c.  Target  Attribute  Vulnerability 

This  factor  affects  the  deterrence  level  of  a  target.  Since  neutral  targets 
are  more  likely  to  be  influenced  by  non-lethal  capabilities,  they  assume  a  higher 
vulnerability  than  suspect  and  hostile  targets.  The  deterrence  level  becomes  a  function  of 
the  target’s  attribute  vulnerability  and  how  often  non-lethal  capabilities  are  employed 
against  them. 

d.  Target  Alpha  Less  Than  Value 

This  factor  captures  the  deterrence  level  threshold  within  the 
PYTHAGORAS  model.  Once  a  target’s  attribute  level,  i.e.  deterrence  level,  depreciates 
below  this  setting,  a  target  will  deter.  Within  a  single  run,  like  threats  will  react  the  same. 
For  example,  in  a  single  run,  all  neutral  targets  will  have  the  same  deterrence  level 
threshold  regardless  of  location  or  speed. 

D.  DESIGN  OF  EXPERIMENT 

This  thesis  seeks  to  utilize  a  space  filling  design  in  order  to  capture  all  possible 
reactions  of  threat  targets  and  effects  of  the  non-lethal  capabilities.  A  space  fdling  design 
refers  to  thorough  coverage  of  the  many  possible  combinations  of  initial  inputs.  This 
fidelity  allows  the  analyst  to  determine  which  factors  have  the  greatest  effect  on  the 
outcomes  with  a  high  level  of  confidence. 

To  completely  cover  a  design  space,  an  analyst  could  use  a  gridded  or  full 
factorial  design.  These  designs  consider  every  possible  combination  of  variable  input 
parameters.  For  this  research,  there  are  a  total  of  33  input  variables  of  interest.  If  the 
scenario  explored  only  two  levels  (high  and  low)  per  variable,  the  simulation  would 
require  2^^  (#  Levels^^  Factors^^  8,589,934,592,  design  points.  If  each  design  point  is 
iterated  a  minimum  of  30  times  for  statistical  validity,  this  design  would  take  4,903 
centuries  to  execute.  Table  3  illustrates  the  number  of  design  points  required  of  different 
gridded  designs. 
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#  factors 

10  Levels 

10'^k  factorial 

5  Levels 

5'^k  factorial 

2  levels 

2^k  factorial 

1 

10 

5 

2 

2 

10^2  =  100 

5^2  =  25 

2A2  =  4 

3 

10^3  =  1,000 

5^3  =  125 

2^3  =  8 

5 

10^5  =  100K 

5^5  =  3,125 

CN 

CO 

II 

LO 

< 

CN 

10 

10^10=  10  billion 

5^10  =  9,765,625 

2^10  =  1,024 

20 

Don’t  even  think 
about  it! 

5^20  =  95  trillion 

2^20  =  1 ,048,576 

40 

5M0  =  9100  trillion  trillion 

2M0  =  1.1  trillion 

Table  3.  Number  of  Simulation  Runs  Required  in  Various  Gridded  Designs  (From 

OA4333,  2006) 

One  alternative  is  to  reduce  the  number  of  input  variables  to  be  explored. 
However,  another  alternative  which  allows  for  the  same  amount  of  input  variables  with 
even  more  levels  per  variable  is  to  use  a  Nearly  Orthogonal  Latin  Hypercube  (NOLH) 
space  filling  design.  Developed  by  LtCol  Thomas  Cioppa,  U.S.  Army,  at  the  Naval 
Postgraduate  School  in  2002,  the  NOLH  facilitates  efficient  exploration  of  a  large  design 
space  by  arranging  factors  in  an  uncorrelated  manner  (see  also  Cioppa  and  Lucas,  2006). 
Applied  to  this  research,  the  NOLH  allows  for  all  33  factors  to  be  explored  on  various 
levels  (up  to  40)  in  only  257  design  points.  Keeping  the  factors  as  uncorrelated  as 
possible  simplifies  the  task  of  linear  regression  modeling  because  it  makes  it  easy  to 
separate  the  impacts  of  different  input  factors  on  the  MOEs.  The  strongest  correlation  in 
the  NOLH  developed  for  this  research  is  -0.08105,  which  is  quite  low.  Figure  12 
depicts  a  scatterplot  matrix  of  the  correlations  between  the  variable  inputs  used  in  this 
research.  Each  tiny  sub-plot  is  a  scatterplot  of  the  levels  of  two  different  variables.  The 
figure  demonstrates  both  the  space  filling  properties  for  quantitative  factors  of  the  NOLH 
and  the  lack  of  correlation  between  the  factors. 
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Figure  12.  Scatterplot  Matrix  of  Factor  Correlations 


E.  DISTRIBUTED  SIMULATION 

In  addition  to  using  efficient  DOEs,  another  way  to  reduce  required  run-time  of  a 
simulation  is  to  network  computers  together  to  distribute  different  parts  of  a  computing 
task  across  individual  processors  (Law  and  Kelton,  2000).  This  thesis  utilizes  distributed 
simulation  in  applying  the  NOLH  DOE  to  the  baseline  scenario. 

1.  Maui  High  Power  Computing  Center  Simulations 

Included  in  the  Project  Albert  suit  of  tools  is  access  to  the  Maui  High  Power 
Computing  Center  (MHPCC),  which  offers  the  ability  to  data  farm  over  a  large  design 
space  in  a  short  amount  of  time.  Using  the  Tiller,  a  program  developed  by  Referenda 
Systems’  Steve  Upton,  the  PYTHAGORAS  scenario  fde  and  a  study  fde  are  generated  in 
extensible  Markup  Language  (XML).  The  study  file  dictates  what  factors  are  to  be 

varied  and  enfolds  the  design  of  experiment  to  be  executed.  MHPCC  takes  these  files, 
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partitions  the  work  over  several  computers,  and  runs  the  experimental  design.  Once  the 
job  is  complete,  MHPCC  personnel  zip  the  output  and  e-mail  it  back  to  the  user  in  one 
consolidated  fde.  This  leads  to  significant  time  savings  over  other  methods  (Michel, 
2006). 

a.  Debugging  Experiment 

The  initial  debugging  design  included  the  factors  from  Table  2  with  two 
exceptions.  First,  the  six  ‘attribute  vulnerability’  factors  were  not  included  (at  the  time, 
an  earlier  version  of  PYTHAGORAS  was  being  used  that  did  not  include  these  modeling 
capabilities).  Second,  the  hierarchy  of  threats  had  not  yet  been  built  into  the  scenario. 
Therefore,  instead  of  three  color  vulnerability  factors  (applied  to  threat  categories  Till), 
there  were  four  factors  (applied  to  the  two  hostile  and  two  neutral  threat  classes).  With 
23  total  factors,  the  NOTH  spreadsheet  tool,  created  by  Dr.  Susan  Sanchez 
(NOLHDesigns.xls,  2006),  generated  a  NOTH  design  yielding  257  design  points  or 
excursions.  Using  the  MHPCC,  each  excursion  was  replicated  30  times,  yielding  7710 
data  points.  This  initial  design  provided  insight  into  the  inner  workings  of  the  model,  the 
NOTH,  and  the  interface  with  MHPCC.  Several  scenario  enhancements  resulted  from 
the  debugging  design. 

b.  Exploration  Experiment 

Prior  to  the  exploration  experiment,  Northrop  Grumman  released  the 
current  version  of  PYTHAGORAS  (Version  1.10).  Included  in  this  update  was  the 
attribute  modeling  capability  (see  Pythagoras  manual  (Bitinas  et  ah,  2006)  for  complete 
list  of  updates  and  new  feature  descriptions).  Table  2  depicts  all  29  factors  applied  to  the 
NOTH  spreadsheet  tool  for  this  design. 

Additionally,  a  technique  called  ‘lockstepping’  was  used  to  alter  several 
factors  and  create  three  new  factors.  Lockstepping  involves  creating  input  parameters  for 
a  variable  from  a  derivative  of  another  variable’s  input  parameters.  For  the  final  design, 
three  lockstepping  processes  were  applied: 

•  The  ‘OD  red  radius  ratio’  and  the  ‘WM  red  radius  ratio’  both  represented 

proportions  of  other  factors.  For  input  into  the  PYTHAGORAS  model, 

both  were  converted  back  to  integer  values. 
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•  The  two  hostile  and  two  neutral  target  color  vulnerability  inputs  were 
derived  from  their  respective  instance,  speed,  and  category-based  color 
vulnerability  input  values. 

•  The  second  hostile  target  group’s  ‘alpha  less  than  value’  was  lockstepped 
to  equal  the  first  hostile  target’s  input  value.  Similarly,  the  second  neutral 
target  and  the  neutral  fishing  target’s  ‘alpha  less  than  value’  were 
lockstepped  to  equal  the  first  neutral  target’s  value. 

The  iteration  level  for  each  excursion  was  increased  to  100,  yielding 
25,700  data  points.  Although  this  design  produced  plenty  of  data  and  some  interesting 
results,  not  all  of  the  MOEs  could  be  extracted  with  enough  fidelity  to  answer  the 
research  questions.  The  PYTHAGORAS  model  allows  for  two  main  types  of  output  from 
each  simulation  run:  1)  end-of-run  MOEs  and  2)  time-series  MOEs.  Due  to  the  high 
volume  of  data  produced  with  the  time-series  MOE’s,  this  time-series  MOEs  are  not 
available  when  using  MHPCC.  The  end-of-run  MOEs  produced  with  this  design  using 
MHPCC  are  suitable  for  initial  analysis  with  all  threats  combined  into  a  single  group. 
However,  this  research  also  calls  for  further  analysis  of  each  individual  threat  class. 
Since  time-series  MOEs  are  available  when  conducting  a  batched  run  locally,  this  author 
decided  to  conduct  a  second  set  of  experiments  in  order  to  capture  the  remaining  MOEs 
of  interest. 

2.  Locally  Paralleled  Simulations 

Within  the  PYTHAGORAS  directory  is  a  fde  called  PythEng.bat.  This  file 
allows  the  user  to  setup  and  execute  multiple  simulation  runs  from  the  command  line  of  a 
local  computer  (Bitinas,  2006).  The  PythEng.bat  fde  allows  the  user  to  output  the  time- 
series  MOE  fdes  in  addition  to  the  end-of-run  MOE  fdes. 

A  scripting  program  written  in  Ruby,  an  object-oriented  programming  language, 
automated  execution  of  the  PythEng.bat  across  each  design  point  (see  Appendix  B).  The 
decision  to  use  Ruby  vs.  other  object-oriented  languages  was  based  on  its  availability  and 
compatibility.  Ruby  is  open  source  and  freely  available  for  both  development  and 
deployment.  Ruby  in  not  restricted  to  a  single  platform  or  vendor,  but  can  be  run  under 
Unix  or  Linux,  Microsoft  Windows,  or  specialized  systems  such  as  BeOS  {What  is  Ruby, 
2006). 
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a.  Test  Experiments 

The  test  experiment  consisted  of  running  the  first  three  design  points  with 
two  iterations.  The  purpose  of  this  small  experiment  was  to  validate  the  sequence  and 
execution  of  the  Ruby  script  designed  for  this  simulation.  After  debugging  for  errors,  the 
experiment  was  run  a  second  time  using  two  computers,  ten  design  points  (or  excursions) 
each  computer,  and  100  iterations  each  excursion.  In  addition  to  successfully  executing 
the  Ruby  scripts,  this  experiment  provided  initial  run-time  performance  for  optimal 
allocation  of  work  during  larger  runs. 

b.  Final  Experiment 

The  full  design  of  33  factors,  257  design  points,  and  100  iterations  per 
design  point  was  run  on  the  same  two  computers.  One  computer  handled  100  design 
points  while  the  second,  newer,  computer  tackled  147.  After  approximately  three  days  of 
continuous  run-time,  the  complete  output  detailing  the  MOEs  on  each  individual  target 
vessel  was  available  and  ready  for  analysis. 

3.  Data  Extrapolation 

When  using  the  time-series  MOE’s,  a  separate  file  is  generated  for  every  run. 
Using  VBA  code  (see  Appendix  C)  and  Microsoft  Excel,  all  25,700  individual  files  were 
compressed  and  concatenated  into  one  composite  data  set. 

F.  SUMMARY 

The  experimental  designs  described  in  this  chapter,  used  in  conjunction  with  the 
computing  power  of  distributed  simulations,  yield  a  large  amount  of  detailed  information 
from  the  simulation  model.  By  implementing  this  analysis  methodology,  the  resultant 
output  is  directly  applicable  in  answering  the  research  questions.  The  next  chapter 
explores  the  analysis  of  this  output,  and  what  insight  it  brings  to  the  questions  at  hand. 
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IV.  DATA  ANALYSIS 


For  in  this  modern  world,  the  instruments  of  warfare  are  not  solely  for 
waging  war.  Far  more  importantly,  they  are  the  means  for  controlling 
peace.  Naval  officers  must  therefore  understand  not  only  how  to  fight  a 
war,  but  how  to  use  the  tremendous  power  which  they  operate  to  sustain  a 
world  of  liberty  and  justice.^ 

Admiral  Arleigh  Burke 

A.  OVERVIEW 

From  scenario  development  to  implementation  of  the  simulation  experimental 
design,  the  previous  chapters  have  focused  on  the  generation  of  data  for  useful  analysis. 
With  a  huge  volume  of  data  now  available,  it  is  time  to  see  what  insight  this  methodology 
produces.  This  chapter  briefly  describes  the  tools  used  for  analysis,  followed  by  a 
detailed  analysis  of  each  MOE  and  threat  class. 

B.  DATA  ANALYSIS  TOOLS 

The  primary  software  used  for  data  analysis  is  JMP  Statistical  Discovery 
Software™.  In  addition  to  its  robust  statistical  applications  and  user  friendliness,  JMP 
dynamically  links  statistics  with  powerful  graphic  visualizations  {JMP,  2006). 

1.  Regression  Models 

a.  Mixed  Stepwise  Regression 

With  33  input  variables,  generating  a  model  including  interactions  for  a 
given  MOE  can  be  tedious.  Therefore,  stepwise  regression  is  used  as  an  approach  to 
selecting  a  subset  of  the  input  variables  when  there  is  little  theory  to  guide  in  the  selection 
of  terms  {Stepwise  Regression,  2006).  The  type  of  stepwise  regression  selected  for  this 
research  is  mixed,  where  the  statistical  software  package  alternates  forward  and  backward 
stepwise  regression  until  the  remaining  terms  are  significant. 


§  Arleigh  Burke,  (1961).  Famous  Navy  Quotes.  Retrieved  September  4,  2006  from 
http://www.history.navy.mil/trivia/trivia02.htm. 
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b.  Standard  Least  Squares  Linear  Regression 

After  stepwise  regression  is  used  to  select  a  model  of  interest,  the  model  is 
fit  to  a  linear  regression  using  standard  least  squares.  Once  fit,  three  statistics  are 
examined  to  determine  the  goodness  and  applicability  of  the  model.  These  are  the 
adjusted  R  ,  the  F-test  statistic,  and  Student’s  t-test  statistic. 

The  adjusted  estimates  the  proportion  of  variation  in  the  response 
variable  that  is  explained  by  the  predictor  variables  and  not  random  ‘noise’  or  error.  The 
purpose  in  using  the  adjusted  R  instead  of  the  regular  R  value  is  to  make  the  model 
more  comparable  against  models  with  a  different  number  of  predictor  variables,  and 
therefore  different  degrees  of  freedom  (A  Multiple  Regression,  2006). 

An  Analysis  of  Variance  (ANOVA)  of  the  data  is  performed  to  determine 
if  the  relationship  between  the  predictors  and  the  response  variable  is  statistically 
significant.  The  resultant  F-test  statistic  and  its  associated  p-value  reveal,  for  a  defined 
confidence  level,  that  at  least  one  of  the  predictors  explains  the  response  variable  better 
than  the  null  model.  For  this  and  all  follow-on  analysis,  the  confidence  level  selected  for 
ANOVA  analysis  is  95%,  meaning  that  when  the  F-test  p-value  is  less  than  0.05,  the 
model  is  statistically  significant. 

Since  the  F-test  does  not  specify  which  of  the  predictor  variables  is 
significant,  a  Student’s  t-test  is  performed  on  each  of  the  variables.  This  determines  if 
the  variable  is  statistically  significant,  within  a  confidence  level,  in  the  presence  of  all 
other  predictor  variables.  The  confidence  level  selected  for  t-tests  conducted  in  this 
research  is  90%.  Therefore,  when  the  t-test  p-value  is  less  than  0.10,  the  associated 
predictor  variable  is  determined  to  be  statistically  significant. 

2.  Recursive  Partitioning 

Recursive  partitioning  refers  to  the  partitioning  of  data  according  to  a  relationship 
between  the  input  variables  and  the  output  variable,  creating  a  ‘tree’  of  partitions. 
Recursive  partitioning  is  not  necessarily  predictive  like  linear  regression,  but  is  an 
effective  data  mining  technique  because  {Recursive  Partitioning,  2006): 
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•  It  is  good  in  exploring  relationships  without  having  a  good  prior  model; 

•  It  handles  large,  complex  problems  easily;  and 

•  The  results  are  very  interpretable. 

The  simulation  model  for  this  research  produces  complex  and  highly  variable 
data.  Therefore,  the  recursive  partitioning  method  effectively  extracts  critical  knowledge, 
insight  and  trend  analysis  in  answering  the  research  questions  where  traditional 
regression  falls  short. 

3.  Data  Visualization 

Several  graphical  visualizations  are  used  to  compliment  and  highlight  the 
statistical  results.  The  following  describes  the  primary  visualization  tools  used  in  this 
research: 

•  Interaction  Profiles  -  In  an  interaction  profile,  non-parallel  lines 
demonstrate  interaction  between  two  variables.  This  provides  a  clear 
visualization  of  both  the  intensity  and  relationship  of  the  interactions; 

•  Pairwise  Plots  -  Pairwise  plots  are  used  frequently  to  display  various 
relationships  between  two  variables;  and 

•  Partition  Trees  -  Partition  trees  display  the  recursive  partitioning  as  a 
decision  tree  with  associated  statistics. 

C.  OVERALL  ANALYSIS 

The  following  analyses  explore  the  three  main  MOEs  for  all  threat  types 
combined. 

1.  MOE  I  -  Deterrence  Ratio 

When  calculating  this  MOE  from  the  generated  data,  the  hostile  threat  classes  are 
not  included.  Since  the  hostile  targets  are  modeled  as  suicide  threats,  ‘deterrence’  of  a 
hostile  threat  is  only  a  temporary  state  before  its  attack  is  resumed.  Therefore,  no  hostile 
threats  will  have  an  ending  status  of  deterred. 

With  the  deterrence  ratio  defined  as  the  response  variable,  stepwise  regression  is 
used  to  identify  the  predictor  variables  of  interest.  The  factors  eligible  for  the  stepwise 
process  are  those  that  represent  controllable  factors.  These  included  ranges  and  firing 
rates  of  the  non-lethal  capabilities  and  the  ROE  postures.  Details  about  the  resultant 
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model  from  the  stepwise  regression,  performed  on  these  seven  factors,  all  two-way 
interactions  between  them,  and  all  quadratic  terms  of  the  main  effects,  are  obtained  via 
the  least-square  linear  regression  module. 


I  Summary  of  Fit 


I  Parameter  Estimates 


RSquare 
RSquare  Adj 
Root  Mean  Square  Error 
Mean  of  Response  0.5901 62 

Observations  (or  Sum  Wgts)  25500 

I  Analysis  of  Variance 

Source  DF  Sum  of  Squares  Mean  Square 

Model  32  348.2894  10.8840 

Error  25467  971.2598  0.0381 

C.  Total  25499  1319.5492 


I  Actual  by  Predicted  Plot 


Overall  Det  Predicted  PO.OOOO 
RSq=0.26  RMSE=0.1953 


2 


Term 

Intercept 

AHDRange 

ODRange 

WMRange 

AHDFirRate 

LDRedRad 

WMRedRad 

LethalRedRad 

(AHDRange-52.4471  )‘(ODRange-35.01 57) 
(AHDRange-52.4471  )*(AHDFirRate-0 .33471 ) 
(AHDRange-52.4471)*(LDRedRad-8.67059) 
(AHDRange-52.4471  )*(VVMRedRad-1 6.5843) 
(AHDRange-52.4471)*(LethalRedRad-32.4235) 
(ODRange-35.01 57)*(AHDFirRate-0 .33471 ) 
(ODRange-35.0157)*(LDRedRad-8.67059) 
(ODRange-35.01 57)‘(WMRedRad-1 6.5843) 
(ODRange-35.0157)‘(LethalRedRad-32.4235) 
(WMRange-l  5.031 4)*(AHDFirRate-0 .33471 ) 
(WMRange-l  5.031 4)*(LDRedRad-8 .67059) 
(WMRange-l  5.031 4)*(WMRedRad-1 6.5843) 
(WMRange-l  5.031 4)‘(LethalRedRad-32.4235) 

( AHDFirRate-0 .33471  )*(LDRedRad-8 .67059) 
(AHDFirRate-0.33471  )*(WMRedRad-1 6.5843) 
(AHDFirRate-0.33471)*(LethalRedRad-32.4235) 
(LDRedRad-8.67059)*(WMRedRad-16.5843) 
(LDRedRad-8.67059)‘(LethalRedRad-32.4235) 
(WMRedRad-16.5843)*(LethalRedRad-32.4235) 
(AHDRange-52.4471  )•(  AHDRange-52.4471 ) 
(ODRange-35.01 57)*(ODRange-35.01 57) 
(AHDFirRate-0.33471  )*(  AHDFirRate-0.33471 ) 
(LDRedRad-8.67059)*(LDRedRad-8.67059) 
(WMRedRad-1 6. 5843)*(WMRedRad-1 6.5843) 
(LethalRedRad-32.4235)*(LethalRedRad-32.4235) 


Estimate  Std  Error 
0.1597881  0.009133 

0.0050627  0.000078 
0.0000798  0.000106 
-0.001739  0.000212 
0.7139239  0.01287 

-0.001426  0.000408 
-0.000309  0.000294 
-0.001124  0.000169 
-0.000026  0.000007 
-0.009843  0.000719 
0.0000368  0.000019 
0.0001613  0.000014 
-0.000139  0.000011 
0.0048122  0.001166 
-0.000083  0.000026 
0.0000863  0.00002 

0.0000686  0.000014 
0.0090491  0.002481 

-0.000222  0.000045 
0.0001514  0.000033 
0.0002639  0.000026 
0.0194532  0.003207 
0.0114066  0.002427 
-0.006534  0.001722 
-0.000414  0.000066 
0.0001052  0.000046 
-0.000248  0.000035 
-0.000054  0.000006 
0.0001225  0.000012 
-1.089064  0.175773 
0.0004826  0.000046 
0.0002541  0.000031 

-0.000042  0.000018 


t  Ratio  Prob>|t| 

17.50  <.0001 

0.7dCQ 45343  3 

55.47  0.0000 

-3.49  0.0005 

-1 .05  0.2925 

-6.66  <.0001 
-3.78  0.0002 

-13.69  <.0001 

1 .90  0.0572 

11.13  <.0001 

-12.89  <.0001 

4.13  <.0001 


4.34^^.^01^3  4 

3.65  0.0003 

-4.89  <.0001 

4.53  <.0001 

10.26  <.0001 
6.07  <.0001 

4.70  <.0001 

-3.79  0.0001 

-6.27  <.0001 

2.28  0.0225 
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Figure  13.  Final  Linear  Regression  Model  on  Overall  Deterrenee  Ratio 


Figure  13  displays  the  statisties  and  model  performanee  values  obtained  from  the 
linear  final  regression  model.  Number  1  in  Figure  13  reports  an  overall  adjusted  R  of 
26.3%.  This  means  that  only  one  quarter  of  the  variability  observed  in  the  deterrence 
ratio  is  explained  by  the  predictors  selected  for  this  model.  A  relatively  low  in  a 
model  that  represents  a  complex  and  highly  variable  real  word  scenario  is  not  surprising. 
Number  2  in  Figure  13  shows,  despite  the  low  R  ,  the  regression  model  is  statistically 
significant  given  the  F-test  p-value.  The  section  labeled  ‘Parameter  Estimates’  in  Figure 
1 3  lists  the  final  factors  selected  as  predictor  variables  for  the  model  and  the  associated  t- 
test  statistic  and  p-value  for  each  predictor.  Of  particular  interest  is  the  t-test  conducted 
on  OD  range.  Number  3  in  Figure  13  shows  OD  range  alone  is  statistically  insignificant 
in  affecting  the  deterrence  ratio.  However,  a  critical  element  of  this  research  is  to  explore 
the  effectiveness  of  the  non-lethal  capabilities  when  used  as  an  element  in  a  system  of 
capabilities.  Number  4  in  Figure  13  exemplifies  that  although  OD  range  alone  is 
statistically  insignificant,  the  interaction  between  OD  range  and  the  ROE  level  for 
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employing  WMs  is  critical.  Number  5  in  Figure  13  shows  that  the  quadratic  terms  for  six 
of  the  seven  main  effects  are  statistically  significant.  Despite  the  statistical  significance 
of  the  quadratic  terms,  including  quadratic  effects  as  potential  explanatory  terms  had  little 
impact  on  the  model’s  explanatory  power. 

Of  the  26  predictor  terms  selected  for  the  model,  19  are  statistically  significant 
interaction  terms.  These  relationships  are  more  clearly  visualized  using  IMF’s 
interaction  profiler,  as  seen  in  Figure  14.  The  former  example  of  interaction  between  OD 
range  and  the  ROE  level  for  WM  employment  is  highlighted  by  Number  1  in  Figure  14. 
This  shows  that  the  effect  of  WM  ROE  is  smaller  when  the  value  for  OD  max  range  is 
low,  but  has  more  effect  on  the  overall  deterrence  ratio  for  higher  values  of  OD  max 
range.  Figure  2  displays  an  opposite  relationship  between  WMRedRad  and 
LethalRedRad.  Here,  the  restrictiveness  of  WM  ROE  has  a  greater  impact  on  the  overall 
deterrence  ratio  when  there  is  a  freer  ROE  for  lethal  force.  When  ROE  for  lethal  force  is 
highly  restrictive,  the  ROE  level  for  WM  employment  has  little  effect  on  the  overall 
deterrence  ratio. 
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Figure  14.  Interaction  Profiles  of  Predictor  Variables 
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In  addition  to  the  information  provided  in  Figure  13,  other  pairwise  plots  are 
available  to  assess  the  performance  of  the  selected  regression  model.  Figure  15  plots  the 
fitted  deterrence  ratio  against  the  model  residuals.  A  primary  assumption  underlying 
linear  regression  is  that  the  residuals  are  homoscedastic,  meaning  that  the  residuals  are 
equally  varied  across  the  observed  values  of  the  response  variable.  If  strong  ‘fanning’ 
trends  are  detected  in  this  plot,  then  there  is  evidence  of  heteroscedasticity,  and  the 
assumption  of  equal  variance  in  the  residuals  is  not  met.  Figure  15  shows  that  there  is  no 
strong  evidence  of  heteroscedasticity  (the  vertical  spread  between  residuals  is  consistent 
across  the  range  of  predicted  deterrence  values)  and  the  modeling  assumption  is  not 
violated. 
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Figure  15.  Plot  of  Residuals  vs.  Fitted  Variables 

Another  assumption  of  linear  regression  modeling  is  normality  in  the  distribution 
of  errors.  Figure  16  portrays  the  use  of  visual  plots  to  determine  the  normality  of  the 
residuals.  The  Normal  Quantile  plot  is  used  to  investigate  whether  the  data  exhibits  the 
standard  bell  curve  of  a  normal  distribution.  Since  the  plotted  deterrence  rate  points 
closely  follow  a  linear  pattern  in  the  Normal  Quantile  plot,  the  assumption  of  normality  is 
not  violated. 
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Figure  16.  Plot  of  Residuals  of  Overall  Deterrence  Data 

Although  the  linear  regression  provides  clear  insight  on  the  importance  of  the 
interaction  between  input  variables,  the  large,  complex  scenario  is  not  otherwise  modeled 
well  by  linear  regression.  Therefore,  we  use  recursive  partitioning  on  the  data.  Unlike 
the  linear  regression  model,  the  partition  tree  for  this  MOE  incorporated  all  33  input 
factors — ^both  the  controllable  factors  (non-lethal  capabilities  ranges  and  firing  rates)  and 
the  uncontrollable  factors  (hostile  and  neutral  behaviors).  The  first  step  is  to  recursively 
‘split’  the  data  to  create  a  partition  tree.  For  each  split,  the  overall  value  of  is 
examined.  From  this  information,  the  point  of  diminishing  returns  was  used  to  define  the 
appropriate  number  of  splits.  The  same  method  is  used  to  determine  splits  for  all 
subsequent  recursive  partitioning. 

Using  four  splits,  the  partition  tree  for  overall  deterrence  rate  yields  an  R^  of 
31.4%.  Although  this  model  captures  more  variability  than  the  linear  regression,  the 
nature  of  the  scenario  continues  to  produce  relatively  low  R  values.  Figure  17  shows  the 
partition  tree  representing  the  recursive  partitioning  for  this  MOE.  Number  1  in  Figure 
17  shows  that  the  most  dominate  factor  in  affecting  the  deterrence  rate  is  the  range  of  the 
AHD.  When  AHD  range  is  greater  than  975  yards  (51  pixels),  the  overall  deterrence 
ratio  is  66%,  a  15%  increase  over  when  the  AHD  range  is  less  than  975  yards.  Figure  2 
shows  that  if  1)  the  AHD  Range  is  less  than  975  yards  and  2)  the  channel  transiting 
neutral  target  has  a  stubborn  deterrence  level  threshold,  then  the  firing  rate  of  the  AHD  is 
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the  next  factor  to  affect  the  deterrence  ratio.  When  each  AHD  broadcast  takes  less  than 


54  seconds  (firing  rate  is  greater  than  .28  shots/time  step),  the  mean  deterrence  rate  is 


47%,  vs.  only  15%  when  the  broadcast  takes  longer. 


1  AHDFirRate<0.28  | 

|AHDFirRate>=0.28  | 

Count 

1300 

Count 

2000 

Mean 

0.1528298 

Mean 

0.4679462 

Std  Dev 

0.197754 

Std  Dev 

0.2279862 

Figure  17.  Partition  Tree  on  Overall  Deterrence  Ratio 


Given  these  findings,  the  primary  insight  gleaned  from  this  MOE  is  that  the  AHD 
has  the  most  impact  in  deterring  targets.  As  this  is  the  initial  non-lethal  capability  used  in 
the  tactic  employed  in  this  scenario,  this  finding  is  generalized  to  say  that  the  first 
response  capability  has  the  most  affect  in  deterring  targets. 

2.  MOE  II  -  Hostile  Identification  Ratio 

As  before,  stepwise  regression  is  used  to  select  the  predictor  variables  of  interest 
given  hostile  identification  ratio  as  the  response  variable.  Number  1  in  Figure  18  shows 
an  adjusted  R  of  only  6.7%.  Again,  although  the  R  is  small.  Number  2  shows  that  the 
linear  regression  model  is  statistically  significant.  Number  3  demonstrates  that 
interaction  and  quadratic  terms  are  equally  as  important  in  this  MOE.  As  before,  the 
assumption  of  homoscedasticity  and  normality  appear  reasonable. 
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Summary  of  Fit 


[  Parameter  Estimate 


RSquare 
RSquare  Adj 
Root  Mean  Square  Error 
Mean  of  Response  0 .51 1 1 6 

Observations  (or  Sum  Wgts)  25700 


I  Analysis  of  Variance 

Source  DF  Sum  of  Squares  Mean  Square 

Model  30  26.80624  0.893541 

Error  25669  363.53894  0.014163 


F  Ratio 
63.0918 
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Term 

Intercept 

AHDRange 

ODRange 

WMRange 

AHDFirRate 

LDRedRad 

WMRedRad 

LethalRedRad 

(AHDRange-52.501 9)*CWMRange-1 5.0078) 
(AHDRange-52.5019)*(AHDFirRate-0.33502) 

( AHDRange-52 .501 9)*(LDRedRad-8 .75097) 
(AHDRange-52.501 9)*CWMRedRad-1 6.6654) 
(AHDRange-52.501 9)*(LethalRedRad-32.501 9) 
(ODRange-35.01 56)*(WMRange-1 5.0078) 
(ODRange-35.0156)*(AHDFirRate-0.33502) 
(ODRange-35.01 56)*(LDRedRad-8 .75097) 
(ODRange-35.01 56)*(WMRedRad-1 6.6654) 
(VVMRange-15.0078)*(AHDFirRate-0.33502) 
(WMRange-l  5 .0078)‘(LDRedRad-8 .75097) 
(WMRange-1 5. 0078)*CWMRedRad-1 6.6654) 
(WMRange-l  5. 0078)‘(LethalRedRad-32. 501 9) 
(AHDFirRate-0.33502)*(LDRedRad-8.75097) 

( AHDFirRate-0 .33502)‘(WMRedRad-1 6 .6654) 
(LDRedRad-8 .75097)*CWMRedRad-1 6 .6654) 
(LDRedRad-8 .75097)*(LethalRedRad-32 .501 9) 
(WMRedRad-l  6.6654)*(LethalRedRad-32.501 9) 
(AHDRange-52.501 9)*(  AHDRange-52 .501 9) 
(ODRange-35.01 56)*(ODRange-35.01 56) 
(LDRedRad-8 .75097)*(LDRedRad-8 .75097) 
(WMRedRad-1 6. 6654)*(WMRedRad-1 6.6654) 
(LethalRedRad-32.501 9)‘(LethalRedRad-32.501 9) 


Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

0.5709612 

0.00552 

103.43 

0.0000 

-0.000592 

0.000047 

-12.60 

<.0001 

0.0001 303 

0.000064 

2.03 

0.0421 

0.0005861 

0.000128 

4.57 

<.0001 

-0.123051 

0.007828 

-15.72 

<.0001 

0.0002061 

0.000246 

0.84 

0.4028 

0.0007038 

0.000178 

3.96 

<.0001 

-0.000078 

0.000102 

-0.77 

0.4438 

-0.000017 

0.000009 

-1 .98 

0.0479 

> 

-0.004429 

0.000433 

-10.23 

<.0001 

0.0000942 

0.000012 

8.11 

<.0001 

-0.000084 

0.000009 

-9.65 

<.0001 

0.0000601 

0.000007 

9.23 

<.0001 

0.0000662 

0.000011 

5.80 

<.0001 

-0.005726 

0.000711 

-8.06 

<.0001 

0.0000348 

0.000015 

2.31 

0.0209 

-0.000042 

0.00001 

-4.20 

<.0001 

0.0226291 

0.001514 

14.95 

<.0001 

0.000124 

0.000027 

4.54 

<.0001 

-0.000081 

0.00002 

-3.99 

<.0001 

-0.000118 

0.000016 

-7.52 

<.0001 

-0.015462 

0.00193 

-8.01 

<.0001 

0.0117848 

0.001291 

9.13 

<.0001 

0.0004643 

0.000039 

11.84 

<.0001 

-0.000242 

0.000027 

-8.93 

<.0001 

0.0001865 

0.000021 

8.83 

<.0001 

-0.000016 

0.000004 

-4.61 

<.0001 

-0.000042 

0.000007 

-6.05 

<.0001 

-0.000346 

0.000028 

-12.57 

<.0001 

-0.00023 

0.000019 

-12.27 

<.0001 

0.0000918 

0.000011 

8.28 

<.0001 

J 

Figure  18.  Final  Linear  Regression  Model  on  Overall  Hostile  Identification  Ratio 
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The  partition  tree  for  this  MOE  sheds  light  on  the  extremely  poor  R  in  the  linear 
regression  model.  Figure  19  illustrates  the  first  three  splits  are  all  factors  that  the  blue 
ship  has  no  control  over,  to  include  hostile  target  speed  and  number  of  suspicious  fishers. 
The  predictor  variables  modeled  in  the  linear  regression  model  do  not  even  appear  in  the 
partition  tree  until  many  successive  splits  later. 


Figure  19.  Partition  Tree  on  Overall  Hostile  Identification  Ratio 
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Number  1  in  Figure  19  shows  the  best  case  scenario,  where  if:  Hostile2  target 
speed  is  greater  than  14  knots  (6  pixels/time  step),  hostile  target  speed  is  greater  than  12 
knots  (5  pixels/time  step),  and  the  number  of  Suspect  Fishers  is  greater  than  2,  then 
targets  are  identified  as  hostile  55%  of  the  time. 

Number  2  in  Figure  19  represents  the  worst  case  scenario.  If  Hostile2  target’s 
speed  is  less  than  14  knots  (6  pixels/time  step),  targets  are  identified  as  hostile  44%  of  the 
time. 

The  difference  between  the  best  case  and  worst  case  scenario  is  only  11%. 
Combined  with  a  lower  of  only  17%,  the  MOE  selected  does  not  appear  to  give  much 
insight  towards  the  research  questions  other  than  the  mean  hostile  identification  ratio. 
Underlying  statistics  such  as  misidentification  rates  are  explored  in  subsequent  analyses. 

3.  MOE  III  -  Warning  Zone  Identification  Ratio 

The  previous  MOE  only  corresponds  to  a  target  identified  as  hostile,  but  not  how 
it  was  identified.  The  scenario  models  two  very  different  ways  by  which  a  target  can  be 
identified  as  hostile.  This  MOE  captures  those  differences  by  recording  the  proportion  of 
targets  identified  outside  of  the  exclusion  zone  (given  that  a  target  is  identified  as 
hostile).  The  linear  regression  shows  an  R  of  26.6%  and  a  statistically  significant  F-test 
p-value.  As  with  the  previous  two  MOE  regressions,  there  is  clear  evidence  that 
interactions  and  quadratic  terms  are  critical.  With  respect  to  the  modeling  assumptions, 
the  model  is  homoscedastic,  but  the  residuals  are  not  normally  distributed.  Instead,  the 
residuals  display  a  strong  right  skew.  Although  this  assumption  is  violated,  it  does  not 
affect  the  regression  because  the  MOE  selected  is  a  mean;  and  therefore  is  not  heavily 
dependent  upon  the  distribution  around  the  statistic.  Figure  20  displays  the  model  results 
in  full  detail. 
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I  Summary  of  Fit 


RSquare 
RSquare  Adj 
Root  Mean  Square  Error 
Mean  of  Response  0.082765 

Observations  (or  Sum  Wgts)  25700 


I  Analysis  of  Variance 

Source  DF  Sum  of  Squares  Mean  Square 

Model  29  156.40682  5.39334 

Error  25670  430.11506  0.01676 

C.  Total  25699  586.52188 


Actual  by  Predicted  Plot 


F  Ratio 
321 .8836 
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Parameter  Estimates 

Term 

Intercept 

AHDRange 

ODRange 

WMRange 

AHDFirRate 

LDRedRad 

WMRedRad 

LethalRedRad 

(AHDRange-52.501 9)*(ODRange-35.01 56) 

I  (AHDRange-52.501 9)*(yVMRange-1 5.0078) 
(AHDRange-52.501 9)*(AHDFirRate-0 .33502) 

( AHDRange-52 .501 9)*(LDRedRad-8 .75097) 
(AHDRange-52.501 9)*(WMRedRad-1 6.6654) 
(AHDRange-52.501 9)*(LethalRedRad-32.501 9) 
(ODRange-35.01 56)*(WMRange-1 5.0078) 
(ODRange-35.01 56)*(VVMRedRad-1 6.6654) 
(ODRange-35.01 56)*(LethalRedRad-32.501 9) 
(WMRange-l  5 .0078)*(  AHDFirRate-0 .33502) 
(WMRange-l  5 .0078)*(LDRedRad-8 .75097) 
(WMRange-l  5.0078)*(WMRedRad-1 6.6654) 
O/VMRange-1 5.0078)*(LethalRedRad-32.501 9) 

( AHDFirRate-0 .33502)*(LDRedRad-8 .75097) 

( AHDFirRate-0 .335O2)*0/VMRedRad-1 6 .6654) 
(LDRedRad-8 .75097)‘(LethalRedRad-32 .501 9) 
(AHDRange-52.501 9)*(  AHDRange-52 .501 9) 
(ODRange-35.01 56)*(ODRange-35.01 56) 
(WMRange-1 5. 0078)*(WMRange-1 5.0078) 

( AHDFirRate-0 .33502)*(  AHDFirRate-0 .33502) 
(LDRedRad-8 .75097)*(LDRedRad-8 .75097) 
(WMRedRad-l  6. 6654)*0/VMRedRad-1 6.6654) 


Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

-0.002473 

0.006049 

-0.41 

0.6827 

0.0014191 

0.000051 

27.91 

<.0001 

0.0011119 

0.00007 

15.95 

<.0001 

0.0002643 

0.000139 

1.90 

0.0580 

0.3176941 

0.008485 

37.44 

<.0001 

-0.006586 

0.000246 

-28.80 

<.0001 

-0.00143 

0.000153 

-9.35 

<.0001 

-0.00231 

0.000102 

-22.75 

<.0001 

0.0000306 

0.000004 
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Figure  20.  Final  Linear  Regression  Model  on  Warning  Zone  Identification  Ratio 


Figure  21  shows  the  corresponding  partition  tree  for  this  MOE.  The  dominant 
input  factor  is  the  speed  of  the  hostile  target.  If  hostile  speed  is  less  than  14  knots  (6 
pixels/time  step),  targets  are  identified  as  hostile  outside  of  the  exclusion  zone,  using  the 
non-lethal  capabilities,  24.6%  of  the  time.  Otherwise,  the  mean  identification  rate  outside 
the  exclusion  zone  is  only  3.8% — a  substantial  difference. 


Figure  21 .  Partition  Tree  on  Overall  Warning  Zone  Identification  Ratio 
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The  linear  regression  models  of  all  three  MOEs  consistently  establish  the 
significance  of  interactions  between  input  factors.  This  illustrates  the  importance  of 
modeling  the  non-lethal  capabilities  as  a  system  and  not  performing  separate  analyses  for 
each  individual  component.  Otherwise,  the  linear  regression  models  produce  little  else  of 
importance.  Therefore,  the  remaining  analyses  will  focus  on  recursive  partitioning  and 
ordinary  statistics.  The  partition  trees  provided  initial  insight  as  to  what  factors  have  the 
most  impact  given  any  threat  faced  by  the  blue  ship. 

D.  ANALYSIS  BY  THREAT  CLASS 

The  following  analyses  explore  the  MOEs  applied  to  each  threat  class.  This 
enables  more  fidelity  in  the  analysis,  resulting  in  more  detailed  findings  in  regards  to 
non-lethal  effectiveness. 

1.  Hostile  Targets 

The  MOE  of  interest  for  specific  analysis  of  the  hostile  threat  class  is  the  MOE 
III,  the  warning  zone  identification  ratio.  Figure  22  displays  the  partition  tree  for  the  first 
hostile  target  class,  which  is  the  group  that  conducts  the  overt  attack  at  scenario  start. 
The  R  for  this  model  is  81.6%,  showing  that  most  of  the  variability  in  the  MOE  is 
explained  with  only  4  input  factors.  The  first  split  occurs  when  the  target  speed  is  14 
knots  (6  pixels/time  step).  This  is  consistent  with  the  initial  findings  of  MOE  3  using  all 
threat  classes.  When  looking  specifically  at  this  threat  class,  the  impact  of  the  inbound 
hostile  speed  is  dramatic.  Number  1  in  Figure  22  shows  the  blue  ship  uses  the  non-lethal 
capabilities  to  identify  the  targets  as  hostile  outside  of  the  exclusion  zone  73%  of  the 
time.  However,  if  the  inbound  targets  are  moving  faster  than  14  knots,  identification  in 
the  warning  zone  occurs  only  6.5%  of  the  time.  This  finding  supports  the  research 
question  of  when  non-lethal  capabilities  are  tactically  appropriate. 

Number  2  in  Figure  22  shows  the  impact  of  the  AHD  firing  rate  when  the  target 
speed  is  less  than  14  knots,  but  ROE  employment  of  the  OD  is  more  restrictive.  In  this 
case,  hostile  identification  outside  the  exclusion  zone  is  improved  by  52%  (from  37%  to 
89%)  when  the  AHD  broadcast  takes  less  than  38  seconds  (firing  rate  is  greater  than  .39 
shots/time  step).  This  provides  valuable  insight  into  the  tactical  implications  when  using 
the  AHD  as  an  immediate  audio  warning  device  versus  as  a  verbal  message  delivery 
system. 
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Number  3  in  Figure  22  exemplifies  the  dominance  of  ROE-type  factors  of  the 
non-lethal  capabilities  factors.  For  both  branches  of  the  partition  tree,  the  next  significant 
factor  after  hostile  target  speed  is  ROE  restrictiveness  of  the  non-lethal  employment. 


Figure  22.  Partition  Tree  on  Hostile  Identification  Ratio  -  Hostile  Target  Class 

The  second  hostile  target  group  represents  the  group  conducting  a  more  covert 
attack  from  a  closer  range  on  the  blue  ship.  Like  the  first  target  group,  the  model  has  a 
reasonably  high  R^  of  64.8%.  Number  1  in  Figure  23  tells  of  comparative  differences 
between  the  two  hostile  threat  types  where  the  only  difference  between  the  two  is 
geography.  The  first  difference  noted  is  the  count  of  observations  observed.  Because  the 
second  hostile  target  group  attacks  the  blue  ship  towards  the  end  of  the  scenario,  there  are 
iterations  when  the  blue  ship  reaches  its  objective  point  before  the  hostile  targets 
intercept.  This  is  primarily  a  function  of  when  the  hostile  targets  detect  the  blue  ship  and 
what  the  hostile  speed  setting  is  for  that  run.  Therefore,  the  data  for  the  partition  tree 
have  been  pre-screened  to  include  only  those  data  points  where  an  interaction  between 
the  hostile  group  and  the  blue  ship  occur. 

The  second  difference  noted  is  that  the  mean  identification  ratio  is  13.8%,  which 
is  7%  lower  than  that  of  the  first  hostile  target  group.  As  expected,  this  demonstrates  that 
geography  impacts  the  effectiveness  of  the  non-lethal  capabilities.  By  conducting  a 
covert  attack  at  a  closer  range,  the  blue  ship  has  less  reaction  time.  Number  2  in  Figure 
23  shows  that  unlike  facing  hostile  targets  in  a  more  open-water  environment,  the 
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dominating  factor  in  constrained  geography  is  the  restrictiveness  of  the  ROE.  For  the 
tactic  modeled,  a  quicker  escalation  to  the  use  of  the  OD  yields  a  25%  difference  in 
hostile  identification  outside  of  the  exclusion  zone.  Number  3  in  Figure  23  shows  that 
the  employment  time  of  the  AHD  is  critical,  just  as  it  was  for  successfully  identifying  the 
first  hostile  target  group. 
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Figure  23.  Partition  Tree  on  Hostile  Identification  Ratio  -  Hostile2  Target  Class 


2.  Neutral  Targets 

The  MOE  of  interest  for  analysis  of  the  neutral  threat  class  is  the  MOE  I,  the 
deterrence  ratio.  Like  the  second  hostile  threat  group,  the  data  were  pre-screened  to 
analyze  only  data  points  where  an  interaction  between  the  neutral  target  and  the  blue  ship 
occur.  Figure  24  displays  the  partition  tree  for  the  channel  transiting  neutral  target.  The 
probability  of  deterrence  for  the  neutral  target  is  92.3%.  When  the  neutral  target’s  speed 
is  less  than  16  knots  (7  pixels/time  step),  the  neutral  target  is  deterred  100%  of  the  time. 
Although  the  regression  tree  does  not  show  dominant  effects  of  the  non-lethal 
capabilities,  this  model  demonstrates  that  positive  action,  through  non-lethal 
employment,  produces  very  high  deterrence  rates  in  neutral  targets  (regardless  of  the 
specific  requirements). 
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Figure  24.  Partition  Tree  on  Deterrence  Ratio  -  Neutral  Target  Class 

In  further  examination  of  the  data,  there  are  27  instances  of  20,037  runs  where  the 
neutral  target  crosses  within  the  blue  ship’s  exclusion  zone  despite  being  deterred.  All  27 
points  originate  from  the  same  design  point,  meaning  that  given  100  iterations  of  the 
same  inputs,  27  runs  resulted  in  this  outcome.  Reasons  for  this  include  maneuvering 
constraints  of  the  neutral  target  and  the  timeliness  of  the  target’s  deterrence.  There  are 
also  four  instances  of  20,037  runs  where  the  neutral  target  is  identified  as  hostile  outside 
of  the  exclusion  zone.  All  four  points  originate  from  the  same  design  point  of  input 
variables.  A  replay  of  these  four  runs  using  PYTHAGORAS’  playback  feature  showed 
that  the  hostile  identification  event  and  the  deterrence  event  happened  at  virtually  the 
same  time.  Of  the  21,569  data  points,  the  blue  ship  identified  the  neutral  target  as  hostile 
only  0.5%  of  the  time  (118  times).  These  cases  were  generally  a  result  of  the  neutral 
target’s  continued  negative  response  to  non-lethal  employment  and  an  unwillingness  to 
deter. 

The  second  neutral  target,  representing  a  crossing  ferry  in  a  geographically 
constrained  area,  also  exemplifies  the  effects  of  geography  on  the  effectiveness  of  the 
non-lethal  capabilities.  Figure  25  shows  a  probability  of  deterrence  of  82.7%,  almost 
10%  less  than  that  of  the  first  neutral  target.  Number  1  in  Figure  25  shows  that  the  factor 
with  the  most  effect  on  the  deterrence  rate  is  the  range  of  the  AHD.  When  the  AHD 
range  is  greater  than  973  yards  (55  pixels),  deterrence  probability  is  94%.  However,  for 
lesser  ranges  of  the  AHD,  the  probability  is  reduced  to  72.7%.  This  result  is  consistent 
with  the  overall  analysis  of  MOE  1,  showing  that  the  requirements  of  the  first  response 
capabilities  are  critical.  This  also  brings  specific  tactical  insight  into  the  use  of  the  AHD. 
If  the  weather  supports  AHD  max  effective  range  past  1,000  yards,  the  AHD  is  a  very 
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effective  first  response  non-lethal  capability.  However,  if  performance  range  of  the  AHD 
is  degraded  due  to  inclement  weather  conditions,  such  as  high  winds,  the  AHD  is  not  as 
effective  as  a  first  response  capability. 


Figure  25.  Partition  Tree  on  Deterrence  Ratio  -  Neutral2  Target  Class 

Like  the  first  neutral  target,  further  examination  of  the  data  is  conducted.  There 
are  460  instances  where  the  neutral  target  crosses  within  the  blue  ship’s  exclusion  zone 
despite  being  deterred.  This  is  respectably  more  than  the  27  observed  with  the  first 
neutral  target  and  is  attributed  to  the  maneuvering  restrictions  of  the  second  neutral 
target.  Despite  being  deterred,  the  ferry  neutral  often  has  no  where  to  run  to  if  caught 
between  the  blue  ship  and  land.  There  are  14  instances  where  the  neutral  target  is 
identified  as  hostile  outside  of  the  exclusion  zone,  also  all  from  the  same  design  point. 
As  before,  a  random  sampling  of  these  14  instances  showed  that  the  hostile  identification 
event  and  the  deterrence  event  happened  at  virtually  the  same  time.  Of  the  25,123  data 
points,  the  blue  ship  identified  the  neutral  target  as  hostile  only  0.7%  of  the  time. 

3.  Loitering  Targets 

The  final  analyses  explore  the  mix  of  two  threat  groups  loitering  in  the  same 
geographic  area,  specifically  a  choke  point.  Figure  26  shows  the  partition  tree  of  the 
deterrence  ratio  of  the  neutral  fishing  targets.  Number  1  in  Figure  26  displays  the  first 
split  in  this  partition.  Unlike  previous  analyses,  the  factor  with  the  most  effect  is  the 
deterrence  threshold  of  the  neutral  fishers.  Number  2  in  Figure  26  shows  when  neutral 
fishers  have  high  deterrence  level  thresholds,  the  factor  of  most  significant  impact  is 
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again  AHD  range.  With  these  slower  moving  targets,  the  performance  threshold  of  the 
AHD  is  750  yards  (43  pixels).  At  ranges  greater  than  750  yards,  the  deterrence  rate  is 
77%,  compared  to  only  37.5%  for  lesser  ranges.  Number  3  in  Figure  26  shows  when 
AHD  range  is  less  than  750  yards,  the  next  critical  factor  is  the  employment  time  of  the 
AHD.  In  this  case,  deterrence  rate  is  improved  49%  (from  11.8%  to  61%)  when  the 
AHD  broadcast  takes  less  than  54  seconds  (firing  rate  is  greater  than  .28  shots/time  step). 
The  last  two  findings  continue  to  emphasize  the  importance  of  the  first  response 
capability  against  neutral  targets. 


Figure  26.  Partition  Tree  on  Deterrence  Ratio  -  Neutral  Fisher  Target  Class 

14.4%  of  the  neutral  fishers  that  are  deterred  cross  within  the  blue  ship’s 
exclusion  zone  despite  being  deterred.  This  is  largely  due  to  the  even  more  restrictive 
geography  combined  with  slower  moving  targets.  There  are  no  instances  where  the 
neutral  fisher  is  both  deterred  and  identified  as  hostile  outside  of  the  exclusion  zone.  Of 
the  25,700  data  points,  the  blue  ship  identifies  the  neutral  fisher  as  hostile  outside  the 
exclusion  zone  only  0.1%  of  the  time. 

Figure  27  shows  the  partition  tree  of  the  deterrence  ratio  of  the  suspect  fishing 
targets.  Like  the  neutral  fishing  targets.  Number  1  in  Figure  26  shows  the  factor  with  the 
most  effect  is  the  deterrence  threshold.  Number  2  depicts  that  when  the  suspect  fishers 
are  easily  deterred,  the  speed  of  the  first  response  capability  is  more  important  than  the 
range.  This  is  expected  since  the  suspect  fishers  do  not  attack  the  blue  ship  until  well 
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within  the  max  range  of  the  AHD.  For  this  situation,  deterrence  rate  is  improved  40% 
(from  10%  to  50%)  when  the  AHD  broadcast  takes  less  than  42  seconds  (firing  rate  is 
greater  than  .36  shots/time  step). 
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Figure  27.  Partition  Tree  on  Deterrence  Ratio  -  Suspect  Fisher  Target  Class 


Overall,  the  suspect  fishers  penetrate  the  blue  ship’s  exclusion  zone  95.4%  of  the 
time.  When  looking  at  MOE  3,  the  blue  ship  identifies  the  suspect  fishers  as  hostile 
outside  of  the  exclusion  zone  only  4  times  of  25,065  observations.  Combined  with  the 
results  of  the  analyses  of  the  neutral  fishers,  two  very  important  conclusions  are  drawn. 
First,  based  on  the  deterrence  rate  of  the  neutral  targets  and  the  hostile  identification  rate 
of  the  suspect  targets,  the  application  of  non-lethal  capabilities  is  extremely  effective  in 
separating  threats  from  non-threats  in  a  homogenous  group  of  threats.  Second,  based  on 
the  percentage  of  both  neutral  and  suspect  fishers  that  penetrate  the  exclusion  zone,  the 
counter-personnel  non-lethal  capabilities  modeled  in  this  research  are  not  sufficient  when 
used  alone  in  order  to  deter  loitering  targets  in  a  chokepoint. 

E.  SUMMARY 

In  this  chapter,  application  of  validated  statistical  tools  has  been  demonstrated  to 
successfully  conduct  data  analysis  on  several  MOE  and  threat  class  combinations.  The 
primary  findings  discovered  from  the  analysis  are  as  follows: 

•  The  employment  of  non-lethal  capabilities  is  extremely  effective  when 
used  to  identify  threats  from  non-threats  in  an  ambiguous  situation. 

•  Inbound  speed  is  the  critical  factor  in  identifying  and  engaging  inbound 
hostile  threats  outside  of  the  exclusion  zone. 

•  The  number  of  inbound  targets  has  little  to  no  impact  on  identification  and 
engagement  rates  of  hostile  targets. 
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•  The  first  response  non-lethal  capability  is  the  most  crucial  in  deterring 
non-suicidal  targets. 

•  The  AHD  is  significantly  more  effective  when  employment  time  is  less 
than  30  seconds  against  hostile  targets  and  1  minute  for  neutral  or  loitering 
targets. 

•  When  used  alone,  counter-personnel  non-lethal  capabilities  fail  to  deter 
loitering  targets  who  attack  when  within  close  proximity 

The  following  chapter  links  these  findings  back  to  the  primary  research  questions 
and  discusses  their  impacts. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


Now  this  is  not  the  end.  It  is  not  even  the  beginning  of  the  end.  But  it  is, 

perhaps,  the  end  of  the  beginning.** 

Sir  Winston  Churchill 

A.  RESEARCH  SUMMARY 

This  research  began  with  questions.  It  is  time  to  return  to  those  questions,  tie  the 
corresponding  analysis  findings  to  them,  and  determine  what  insight  can  be  gained.  The 
research  questions  originally  stated  in  Chapter  1  are  reproduced  below,  but  now  followed 
by  the  applicable  results  obtained  from  this  research. 

What  non-lethal  capabilities  are  required  in  a  maritime  force  protection 
environment  in  order  to  effectively  determine  intent  and/or  deter  suspicious  small 
vessels?  This  research  explored  this  question  against  a  multitude  of  threat-types, 
different  geographies,  and  varying  conditions.  As  such,  no  singular  answer  is 
appropriate.  Instead,  multiple  insights  are  gained  for  all  possible  environments. 
Therefore,  this  primary  question  is  addressed  in  each  of  the  follow-on  discussions  to  the 
sub-questions  below. 

In  what  region  of  conditions  are  non-lethal  capabilities  tactically  appropriate? 
The  effectiveness  rates  of  identification  and  deterrence  are  highly  sensitive  to  the 
application  of  the  first  response  capability  and  the  timeliness  in  employment  of 
subsequent  capabilities.  If  the  speeds  of  the  incoming  targets  are  less  than  14-16  knots, 
the  tactic  modeled  is  effective.  A  39%  increase  in  effectiveness  is  observed  in  the  worst 
case  scenario  (suicidal  hostile  target  in  a  geographically  constrained  region)  when  the 
employment  time  of  the  AHD  is  kept  below  30  seconds.  When  inbound  speeds  are 
higher,  alternate  tactics  and  capabilities  must  be  explored. 

What,  if  any,  are  the  major  differences  given  varying  geographic  constraints? 
For  both  hostile  and  neutral  targets,  constrained  geography  resulted  in  lower 
identification  and  deterrence  rates.  Any  warfighter  knows  that  limited  geography 

**Winston  Churchill,  (1942).  Classic  Quotes.  Retrieved  September  7,  2006  from 
http://www.quotationspage.com/quote/2492 1  .html. 
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translates  into  reduced  prosecution  time.  The  specific  insight  gained  through  this 
research  for  a  constrained  geography  environment  is: 

•  The  effectiveness  of  the  AHD  must  exceed  975  yards. 

•  Effectiveness  is  extremely  dependent  on  a  relaxed  ROE  for  the  OD  and 
WM. 

•  If  these  two  conditions  cannot  be  met,  a  different  tactic  is  required. 

What  requirements  are  needed  of  non-lethal  capabilities  in  order  to  effectively 
separate  threats  from  non-threats  in  a  homogeneous  group?  The  first  necessary  task 
when  facing  a  group  of  mixed  targets  is  to  separate  the  threats  from  the  non-threats. 
Within  the  loitering  targets,  the  neutral  targets  were  identified  as  non-threats  74%  of  the 
time  while  the  suspect  fishers  were  recognized  as  threats  95.4%  of  the  time.  The  most 
critical  factor  in  achieving  these  levels  of  effectiveness  was  simply  the  use  of  the  non- 
lethal  capabilities  in  accordance  with  the  modeled  tactic.  Meaning,  the  specific 
requirements  of  the  non-lethal  capabilities  did  not  cause  drastic  variations  in  the  results. 
The  fact  that  proactive  non-lethal  force  was  applied,  regardless  of  specific  ranges  and 
firing  rates,  proved  extremely  effective  in  separating  threats  from  non-threats. 

Conversely,  the  sole  use  of  counter-personnel  non-lethal  capabilities  in  a  choke 
point  filled  with  multiple  threat  types  did  not  prove  effective  for  deterrence.  Neutral 
fishers  were  deterred  80.6%  of  the  time,  however  this  dropped  significantly  to  37.5% 
when  the  AHD  range  was  less  than  750  yards.  A  critical  finding  was  that  if  the  AHD 
range  is  less  than  750  yards,  the  firing  rate  should  be  kept  below  one  minute  in  order 
produce  the  highest  possible  deterrence  rate.  Deterrence  of  the  suspect  fishers,  which  are 
the  vital  vessels  to  discourage,  occurred  only  5.4%  of  the  time.  Not  surprisingly,  this 
result  mandates  the  necessity  of  picket  boats  and  counter-materiel  non-lethal  capabilities 
for  this  environment. 

Is  a  multi-agent  simulation  (MAS)  an  appropriate  means  to  represent  and  analyze 
a  ship’s  ability  to  determine  intent  and  use  continuum  of force?  Yes.  The  flexibility 
inherent  in  MAS,  and  particularly  within  the  PYTHAGORAS  model,  has  provided  the 
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framework  necessary  to  answer  the  previous  questions.  PYTHAGORAS’  capacity  to 
quantify  and  model  elements  such  as  ROE,  fear,  aggressiveness  and  influence,  has  been 
crucial  in  producing  useful  and  applicable  insight  for  a  complex  problem. 

B.  POTENTIAL  FOLLOW-ON  RESEARCH 

As  shown  in  this  research  MAS,  efficient  experimental  designs  and  high  power 
computing  are  appropriate  for  a  broad  scope  of  analysis  in  the  realm  of  non-lethal 
capabilities  and  maritime  force  protection.  The  following  recommendations  are  provided 
for  areas  of  research  which  could  benefit  from  the  data  farming  methodology  applied  to 
this  thesis. 

1.  Requirements 

This  thesis  identified  situations  when  targets  are  not  effectively  deterred  using 
only  counter-personnel  non-lethal  capabilities.  Current  doctrine  employs  small  boats  to 
be  used  as  pickets  to  physically  deter  inbound  vessels.  Additionally,  counter-materiel 
non-lethal  capabilities  are  also  being  developed  in  the  maritime  environment.  This 
research  successfully  modeled  non-lethal  capabilities  modeled  as  a  system. 
Recommended  future  efforts  would  be  the  expansion  of  this  system  using  the  same 
scenario  to  include  counter-materiel  capabilities  and  picket  boats. 

2.  Tactics 

One  specific  tactic,  adapted  from  current  Navy  tactics,  was  modeled  in  the 
scenario.  This  research  identified  performance  conditions  in  which  the  modeled  tactic 
was  not  effective.  The  methodology  used  in  this  research  can  easily  be  used  to  conduct  a 
comparative  analysis  of  multiple  tactics.  By  creating  a  set  of  scenarios  where  each 
represents  a  unique  tactic,  the  scenarios  can  be  compared  to  determine  the  most  effective 
tactic(s)  for  each  situation. 

3.  Vulnerability  Assessments 

As  demonstrated,  geography  is  a  critical  factor  in  the  force  protection  of  the  blue 
ship.  This  methodology  allows  a  port  or  chokepoint  to  be  exhaustively  searched  to 
determine  the  most  sensitive  geographical  points  of  vulnerability.  Since  real  world  maps, 
charts  or  satellite  photos  can  be  overlaid  into  the  PYTHAGORAS  model,  construction  of 
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a  vulnerability-based  scenario  for  any  place  in  the  world  can  be  accomplished  quickly. 
Future  work  should  include  application  of  this  methodology  to  assist  planners  in  port 
integrated  vulnerability  assessments  prior  to  a  ship’s  arrival. 

4.  Alternate  Mission  Objectives 

The  use  of  non-lethal  capabilities  is  by  no  means  limited  to  an  entering  port  force 
protection  scenario.  The  application  of  this  methodology  would  be  extremely  relevant 
for  analysis  of  alternate  missions,  such  as  straits  transits  and  maritime  interdiction 
missions. 

5.  Refined  Input  Parameters 

As  seen  in  the  deterrence  of  the  neutral  fishers,  the  most  significant  factor  was  the 
deterrence  level  threshold,  or  willingness  of  the  targets  to  deter.  Since  no  data  exists  on 
these  values,  the  experiment  ranged  across  a  wide  variety  of  possibilities.  As  non-lethal 
capabilities  become  fielded,  and  real-world  data  is  collected  on  these  behavioral 
variables,  future  analyses  can  incorporate  more  refined  behavior  inputs  or  explore  the 
reasonableness  of  these  input  parameter  ranges. 

C.  RESEARCH  IMPACT 

Using  the  data  farming  methodology  to  answer  the  research  questions  outlined  in 
this  thesis  is  only  the  first  step.  The  next  level  of  insight  is  reached  by  understanding  the 
applicability  of  this  research.  The  following  outlines  the  impact  of  this  research  and 
follow-on  initiatives  in  the  fielding  of  non-lethal  capabilities  for  maritime  missions. 

Focuses  Capability  Requirements  -  This  research  uses  analytic  rigor  to 
compliment  subject  matter  expertise  in  effectively  defining  requirements  for  three 
pending  maritime  capabilities.  Moreover,  this  research  is  the  first  to  evaluate  the 
requirements  needed  for  an  effective  system  of  complimentary  non-lethal  capabilities. 

Validates  Tactics  -  Current  Navy  Tactics,  Techniques  and  Procedures  for  anti¬ 
terrorism/force  protection  include  only  current  capabilities.  This  research  analyzes  the 
integration  of  imminent  capabilities  into  current  tactics  prior  to  acquisition,  thereby 
providing  warfighters  with  updated  tactics  as  soon  as  capabilities  are  released.  Included 
is  an  analysis  of  tactical  effectiveness  given  varying  rules  of  engagements,  covering  any 
geo-political  situation. 
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Revolutionizes  Vulnerability  Assessments  -  Navy  doctrine  requires  examination 
of  likely  enemy  courses  of  action  and  friendly  counteraction  when  conducting  anti¬ 
terrorist/force  protection  planning.  Current  tactical-level  wargaming  efforts  include  table 
top  discussions,  which  generally  cover  only  a  few  possible  courses  of  action.  This  thesis 
leverages  high  power  computing  to  evaluate  virtually  any  possible  interaction  with 
multiple  threat-types,  thereby  generating  a  robust  port  vulnerability  assessment.  The 
methodology  and  model  used  in  this  research  are  quickly  altered  to  support  any  port  or 
choke  point  geography. 

In  summary,  this  research  examines  newly  fielded  or  not-yet-fielded  technologies, 
meaning  little  performance  data  exists.  Secondly,  this  research  tackles  complex 
interactions,  including  critical  human  factors,  in  emulating  an  intricate,  real-world 
scenario.  Lastly,  this  research  attempts  to  capture  and  make  sense  of  the  multitude  of 
possible  outcomes.  George  E.  P.  Box  said,  “All  models  are  wrong,  but  some  are  useful” 
(Box,  1979).  Despite  the  initial  limitations  and  complexities,  this  thesis  produces 
valuable  insights  through  application  of  proven  operations  research  tools  and  techniques. 
However,  this  is  nothing  but  a  beginning.  Through  continued  application  of  this  work 
across  the  spectrum  of  non-lethal  applications  in  maritime  operations,  we  considerably 
enhance  our  efforts  to  preserve  the  freedom  of  the  seas  and  ultimately,  to  bring  our  sailors 
safely  home. 
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APPENDIX  A.  PYTHAGORAS  COLOR  COMPARISON  TOOL 


PYTHAGORAS  uses  red-green-blue  (RGB)  color  spectrum  to  differentiate 
‘sidedness’  or  affdiations  in  a  given  scenario  (see  the  PYTHAGORAS  manual  for  further 
detail).  This  approach  provides  nearly  unlimited  flexibility  in  assigning  multiple-sided 
forces  and  their  relationships  to  one  another.  Because  a  user  can  quickly  lose  track  of 
who  is  affdiated  with  whom,  the  PYTHAGORAS  software  package  includes  a  Microsoft 
Excel  document  that  allows  the  user  to  create  the  affdiations  as  he  or  she  would  in  the 
PYTHAGORAS  GUI,  and  then  quickly  determine  the  resultant  relationships  based  on 
these  inputs. 
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Figure  28.  Sidedness  RGB  Inputs  for  PYTHAGORAS  Scenario 


Figure  28  displays  the  sidedness  RGB  levels  inputted  into  the  PYTHAGORAS 
scenario  for  this  research.  The  first  four  agents  listed  represent  the  three  non-lethal 
capabilities  and  the  lethal  capability  of  the  blue  ship.  The  remaining  three  listed  agents 
do  not  represent  an  actual  target,  but  an  identification  state  of  an  inbound  target. 
Independent  of  its  ground  truth  identification,  the  blue  ship  will  always  hold  targets  in 
one  of  these  three  states. 
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Figure  29.  Agent  Affiliations  for  PYTHAGORAS  Scenario 


Given  the  inputs,  the  PYTHAGORAS  color  comparison  tool  automatically 
generates  the  associations  of  the  listed  agents  to  one  another.  Figure  29  displays  the 
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affiliations  of  this  thesis.  An  agent  can  ‘see’  another  agent  in  one  of  four  ways:  Unit, 
friend,  enemy  or  neutral.  In  this  example,  the  AHD  and  OD  see  all  targets  as  enemy, 
meaning  all  targets  are  eligible  to  be  engaged  with  AHD  and  OD  non-lethal  capabilities. 
WMs  will  only  be  employed  against  targets  identified  as  suspect  or  hostile,  while  lethal 
force  will  only  be  used  against  hostile  targets. 

The  targets  view  all  blue  assets  as  ‘enemy.’  This  does  not  mean  that  every  target 
type  attacks  the  blue  ship.  Several  subsequent  decision  rules  invoked  when  an  agent 
interacts  with  another  agent  are  based  on  the  affiliation  between  the  two.  Therefore,  the 
categorization  of  ‘enemy’  is  used  to  define  subsequent  sensing  and  movement  triggers. 
For  example,  when  a  neutral  target  is  deterred,  its  movement  desire  is  changed  to  move 
away  from  the  nearest  ‘enemy,’  which  is  defined  in  Figure  29  as  the  blue  ship.  The  blank 
spaces  relate  to  an  affiliation  of  neutral.  The  neutral  association  of  all  targets  to  one 
another  is  based  on  the  initial  assumption  that  all  agent  classes  move,  decide  and  engage 
independently  from  one  another. 
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APPENDIX  B.  RUBY  SCRIPT  FOR  DATA  FARMING 


As  discussed  in  Chapter  III,  Ruby  scripting  code  is  used  to  implement  the  NOLH 
DOE  in  a  distributed  simulation.  The  script  serves  to  aceomplish  5  main  tasks: 

1)  Take  the  appropriate  design  point  from  the  NOLH.xls  spreadsheet. 

2)  Update  the  baseline  PYTHAGORAS  XML  seenario  file  with  the  current 
design  point  values,  taken  from  the  NOLH.xls  spreadsheet. 

3)  Update  a  text  input  fde  to  be  used  to  exeeute  the  Pythagoras  batch  engine.  The 
input  file  includes  all  information  needed  to  run  the  Pyth.Eng.bat  including  seed  number, 
index  number,  and  output  storage  location. 

4)  Execute  the  design  point  in  PYTHAGORAS  for  the  desired  number  of 
iterations  (designated  in  the  input  fde)  using  the  Pyth.Eng.bat  fde  ineluded  in  the 
PYTHAGORAS  fdes. 

5)  Repeat  the  process  for  the  next  design  point. 

The  speeifie  code  written  to  aceomplish  this  is  provided  below: 


Line 

Code 

1 

File  Name:  FarmPyth.rb 

2 

#Add-in's  required  to  manipulate  XML  files  with  Ruby 

3 

require  "rexml/document” 

4 

include  REXML 

5 

#Method  to  update  the  XML  scenario  file  with  a  design  point.  The  input  parameters  for  the 

6 

current  design  point  are  passed  to  the  method  in  the  valueArray.  Each  factor  is  represented  by 

7 

an  element  in  the  array.  The  xpath  in  each  [  ]  set  represents  the  xpath  to  the  specific  factor  in 

8 

the  Pythagoras.xml  scenario  file.# 

9 

def  updateScenarioF  ile(valueArray) 

10 

#loads  the  sceanio  and  converts  it  to  REXML  for  manipulation 

11 

baseScenarioFile  =  File.new('E:\Norfolk.xmr,  ’’r”) 

12 

baseScenarioDoc  =  Document. new(baseScenarioFile) 

13 

1  A 

baseScenarioFile. close 

15 

#updates  AHDRange 

16 

baseScenarioDoc. elements['’pythagoras/weaponList/weapon/maxRange”]. text  = 

17 

valueArray  [0] 

18 

#updates  OD  Range 

19 

baseScenarioDoc. elements['’pythagoras/weaponList/weapon[2]/maxRange"].text  = 

20 

value  Array[l] 
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Line 

Code 

21 

File  Name:  FarmPyth.rb 

22 

#Add-in's  required  to  manipulate  XML  files  with  Ruby 

23 

require  ’’rexml/documenf  ’ 

24 

include  REXML 

25 

#Method  to  update  the  XML  scenario  file  with  a  design  point.  The  input  parameters  for  the 

26 

current  design  point  are  passed  to  the  method  in  the  valueArray.  Each  factor  is  represented  by 

27 

an  element  in  the  array.  The  xpath  in  each  [  ]  set  represents  the  xpath  to  the  specific  factor  in 

28 

the  Pythagoras.xml  scenario  file.# 

29 

def  updateScenarioF  ile(valueArray) 

30 

#loads  the  sceanio  and  converts  it  to  REXML  for  manipulation 

31 

baseScenarioFile  =  File.new('E:\Norfolk.xmr,  "r") 

32 

baseScenarioDoc  =  Document.new(baseScenarioFile) 

33 

baseScenarioFile. close 

35 

#updates  AHDRange 

36 

baseScenarioDoc. elements['’pythagoras/weaponList/weapon/maxRange'’]. text  = 

37 

valueArray[0] 

38 

#updates  OD  Range 

39 

baseScenarioDoc. elements[”pythagoras/weaponList/weapon[2]/maxRange'’]. text  = 

40 

valueArray[l] 

41 

#updates  WM  Range 

42 

baseScenarioDoc. elements[”pythagoras/weaponList/weapon[5]/maxRange'’]. text  = 

43 

valueArray[2] 

44 

#updates  AHD  Firing  Rate 

45 

baseScenarioDoc. elements['’pythagoras/weaponList/weapon/shotsPerTimeStep'’]. text  = 

46 

valueArray  [3] 

47 

#updates  OD  Red  Radius 

48 

baseScenarioDoc. elements['’pythagoras/sidednessList/sidedness[2]/enemy/radius'’]. text  = 

49 

valueArray[4] 

50 

#updates  WM  Red  Radius 

51 

baseScenarioDoc. elements['’pythagoras/sidednessList/sidedness[5]/enemy/radius'’]. text  = 

52 

value  Array  [5] 

53 

#updates  Lethal  Red  Radius 

54 

baseScenarioDoc. elements[”pythagoras/sidednessList/sidedness[3]/enemy/radius”].text  = 

55 

valueArray[6] 

56 

#updates  TargetHostile  Color  Vulnerability 

57 

baseScenarioDoc. elements['’pythagoras/agentList/agent[6]/colorVulnerability/colorVulnerabilit 

58 

y  Value"],  text  =  valueArray  [7] 

59 

#updates  TargetHostile [2]  Color  Vulnerability 

60 

baseScenarioDoc. elements["pythagoras/agentList/agent[7]/colorVulnerability/colorVulnerabilit 

61 

yValue"].text  =  valueArray[8] 

62 

#updates  TargetNeutral  Color  Vulnerability 

63 

baseScenarioDoc. elements["pythagoras/agentList/agent[8]/colorVulnerability/colorVulnerabilit 

64 

yValue"].text  =  valueArray[9] 

65 

#updates  TargetNeutral[2]  Color  Vulnerability 

66 

baseScenarioDoc. elements["pythagoras/agentList/agent[9]/colorVulnerability/colorVulnerabilit 

67 

y  Value"],  text  =  valueArray  [10] 

68 

#updates  TargetNeutralFisher  Color  Vulnerability 

69 

baseScenarioDoc. elements["pythagoras/agentList/agent[10]/colorVulnerability/colorVulnerabil 

70 

ity  Value"]. text  =  valueArray[l  1] 

71 

#updates  TargetSuspectFisher  Color  Vulnerability 

72 

baseScenarioDoc. elements["pythagoras/agentList/agent[  1 1  ]/colorVulnerability/colorVulnerabil 

73 

ity  Value"]. text  =  valueArray[12] 
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Line 

Code 

74 

#updates  TargetHostile  Attribute  Vulnerability 

75 

baseScenarioDoc.elements['’pythagoras/agentList/agent[6]/attributeVulnerability/attributeVuln 

76 

erability  Value"],  text  =  valueArray[13] 

77 

#updates  TargetHostile [2]  Attribute  Vulnerability 

78 

baseScenarioDoc.elements["pythagoras/agentList/agent[7]/attributeVulnerability/attributeVuln 

79 

erabilityValue"].text  =  valueArray[14] 

80 

#updates  TargetNeutral  Attribute  Vulnerability 

81 

baseScenarioDoc.elements["pythagoras/agentList/agent[8]/attributeVulnerability/attributeVuln 

82 

erability  Value"],  text  =  valueArray[15] 

83 

#updates  TargetNeutral[2]  Attribute  Vulnerability 

84 

baseScenarioDoc.elements["pythagoras/agentList/agent[9]/attributeVulnerability/attributeVuln 

85 

erability  Value"],  text  =  valueArray[16] 

86 

#updates  TargetNeutralFisher  Attribute  Vulnerability 

87 

baseScenarioDoc.elements["pythagoras/agentList/agent[10]/attributeVulnerability/attributeVul 

88 

nerability  Value"],  text  =  valueArray[17] 

89 

#updates  TargetSuspectFisher  Attribute  Vulnerability 

90 

baseScenarioDoc.elements["pythagoras/agentList/agent[ll]/attributeVulnerability/attributeVul 

91 

nerabilityValue"].text  =  valueArray[18] 

92 

#updates  TargetHostile  Instance 

93 

baseScenarioDoc.elements["pythagoras/agentList/agent[6]/instance"].text  =  valueArray[19] 

94 

#updates  TargetHostile [2]  Instance 

95 

baseScenarioDoc.elements["pythagoras/agentList/agent[7]/instance"]  .text  =  valueArray  [20] 

96 

#updates  TargetNeutralFisher  Instance 

97 

baseScenarioDoc.elements["pythagoras/agentList/agent[10]/instance"].text  =  valueArray[21] 

98 

#updates  TargetSuspectFisher  Instance 

99 

baseScenarioDoc.elements["pythagoras/agentList/agent[l  l]/instance"].text  =  valueArray[22] 

100 

#updates  TargetHostile  Speed 

101 

baseScenarioDoc.elements["pythagoras/agentList/agent[6]/speed/avgSpeed"].text  = 

102 

valueArray  [23] 

103 

#updates  TargetHostile [2]  Speed 

104 

baseScenarioDoc.elements["pythagoras/agentList/agent[7]/speed/avgSpeed"].text  = 

105 

valueArray  [24] 

106 

#updates  TargetNeutral  Speed 

107 

baseScenarioDoc.elements["pythagoras/agentList/agent[8]/speed/avgSpeed"].text  = 

108 

valueArray  [2  5] 

109 

#updates  TargetNeutral[2]  Speed 

110 

baseScenarioDoc.elements["pythagoras/agentList/agent[9]/speed/avgSpeed"].text  = 

111 

valueArray  [2  6] 

112 

#updates  TargetHostile  Alpha  Less  Than  Trigger  Value 

113 

BaseScenarioDoc.elements["pythagoras/agentList/agent[6]/triggerList/alphaLossTrigger/lessT 

114 

han"].text  =  valueArray  [27] 

115 

#updates  TargetHostile [2]  Alpha  Less  Than  Trigger  Value 

116 

baseScenarioDoc.elements["pythagoras/agentList/agent[7]/triggerList/alphaLossTrigger/lessT 

117 

han"].text  =  valueArray  [2  8] 

118 

#updates  TargetNeutral  Alpha  Less  Than  Trigger  Value 

119 

baseScenarioDoc.elements["pythagoras/agentList/agent[8]/triggerList/alphaLossTrigger/lessT 

120 

han"].text  =  valueArray  [29] 

121 

#updates  TargetNeutral [2]  Alpha  Less  Than  Trigger  Value 

122 

baseScenarioDoc.elements["pythagoras/agentList/agent[9]/triggerList/alphaLossTrigger/lessT 

123 

han"].text  =  valueArray  [30] 

124 

#updates  TargetNeutralFisher  Alpha  Less  Than  Trigger  Value 

125 

baseScenarioDoc.elements["pythagoras/agentList/agent[10]/triggerList/alphaLossTrigger/less 

126 

Than"],  text  =  valueArray[31] 

61 


Line 

Code 

111 

#updates  TargetSuspectFisher  Alpha  Less  Than  Trigger  Value 

128 

baseScenarioDoc.  elements  ['’pythagoras/agentList/agent[  1 1  ]/triggerList/alphaLossTrigger/less 

129 

Than”]. text  =  valueArray[32] 

130 

#writes  the  changes  to  the  .xml  file  for  implementation 

131 

file  =  File.new('E:\Norfolk.xmr,  ”w”) 

132 

baseScenarioDoc.write(file) 

133 

file. close 

134 

end 

135 

#Opens  the  Design  of  Experiment  file  and  strips  the  factor  names  from  1  st  row 

136 

SinputFile  =  File.new('C:\Pythagoras\inputData.csv',  ”r”) 

137 

ScolumnHeaders  =  SinputFile.readline. strip. split(”,”) 

138 

#Loop  to  iterate  through  the  design  points.  'X'. times  represents  the  number  of  design  #points 

139 

to  be  run. 

140 

lOO.times  do  designPoint 

141 

#Sets  beginning  seed  and  index  for  current  design  point 

142 

$seed  =  (designPoint*  100)  +  1 

143 

$index  =  (designPoint*  100)  +  1 

144 

#Opens  old  pythenginput  file  to  change  seed  and  index.  The  pythinput  file  feeds  the 

145 

#PythEng.bat  file  the  necessary  inputs  to  run. 

146 

pythenginput  =  File.open('C:\Pythagoras\pythenginput.txf ,  ”r+”) 

147 

array  =  pythenginput.readlines 

148 

pythenginput.  close 

149 

array  [2]  =  ”#{$seed}\n” 

150 

array  [3]  =  ”#{$index}\n” 

151 

#Writes  new  pythenginput  file  for  current  design  point 

152 

logstream  =  File.new('C:\Pythagoras\pythenginput.txf ,  ”w”) 

153 

7. times  { i  logstream. write(array[i])  } 

154 

logstream.  close 

155 

#Updates  XML  scenario  file  with  current  design  point  input  parameters 

156 

valueArray  =  SinputFile.readline. strip. split(”,”) 

157 

updateScenarioF  ile(valueArray) 

158 

puts  ’’Attempting  to  calculate  designPoint  ”  +  designPoint.to  s 

159 

#executes  Pythagoras  (both  pytheng.bat  and  pythenginput.txt  must  be  in  same  folder  as 

160 

#FarmPyth.rb) 

161 

puts  'PythEng.bat  <  pythenginput.txt' 

162 

163 

print  ’’Just  finished  ”  +  designPoint.to_s 

164 

end 

165 

print  ’’Batch  Complete\n” 
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APPENDIX  C.  VBA  SCRIPT  FOR  DATA  CLEANING 


As  discussed  in  Chapter  3,  a  sizable  amount  of  data  cleaning  was  needed  in  order 
to  produce  a  single  file  to  be  used  for  data  analysis.  The  following  sub-programs  coded 
using  VBA  and  Microsoft  Excel  macro  recording  accomplish  the  required  data  cleaning 
for  this  research. 

DPIterateO:  This  class  takes  the  100  design  points  of  the  NOLH.xls  spreadsheet 
and  creates  blank  rows  in  between  each  design  point  which  correspond  to  the  number  of 
desired  iterations  per  design  point. 

Sub  DPIterateO 

1 

’  DPIterate  Macro 

’  Macro  recorded  8/23/2006  by  Lisa  R.  Sickinger 

'  Creates  rows  in  between  each  design  point  for  iterations 

f 

For  i  =  0  To  256 

Cells(((100  *  i)  +  2),  2). Select '  TOO'  is  number  of  iterations  per  run 
For  j  =  0  To  98  '  '98'  is  number  of  iterations  per  run  -  2 
S  election .  EntireRow.  Insert 
Next  j 

Next  i 

End  Sub 


Rovs^CopyQ:  This  copies  the  input  data  from  each  design  point  to  the  nevs^ly 
created  blank  rows  associated  with  that  design  point. 

Sub  RowCopyO 

'  RowCopy  Macro 

'  Macro  recorded  8/23/2006  by  Lisa  R.  Sickinger 
'  Copies  each  design  point  applicable  number  of  times 

For  i  =  0  To  256 

Range((Cells(((100  *  i)  +  2),  2)),  (Cells(((100  *  i)  +  2),  34))).Select 
"100'  =  number  of  design  points 
"34'  =  number  of  factors  +  1 

Selection. AutoFill  Destination:=Range((Cells(((100  *  i)  +  2),  2)),  (Cells(((100  *  i)  +  101),  34))), 
Type:=xlFillDefault 

"101'  =  number  of  design  points  +  1 

Range((Cells(((100  *  i)  +  2),  2)),  (Cells(((100  *  i)  +  101),  34))).Select 
Next  i 
End  Sub 


63 


ConsolidateO:  This  class  takes  the  25,700  individual  AGB  files  created  from  the 
time  series  MOEs  and  consolidates  them  to  257  files,  one  for  each  design  point.  An 
individual  file  has  a  header  with  agent  information,  followed  by  100  rows  of  raw  data, 
corresponding  to  the  output  data  of  the  100  iterations  for  that  design  point.  Because 
different  design  points  have  different  number  of  agents,  the  program  is  flexible  to 
construct  the  correct  number  of  columns  for  each  design  point  file. 

Sub  ConsolidateO 

'  Consolidate  Macro 

'  Macro  recorded  8/20/2006  by  Lisa  Sickinger 

'  change  name  of  file  to  reflect  file  that  starts  current  design  point 
begDP  =  Cells(3,  3)  ‘  first  dp 
endDP  =  Cells(4,  3)  ‘  last  dp 
For  dp  =  begDP  To  endDP 
Name  =  (dp  *  100)  -  99 
Workbooks. Open  Filename :=  _ 

”C:\Documents  and  Settings\SEEDLab\NorfolkMOE\DP'’  &  dp  &  ’’Output.xls” 

Workbooks.  OpenText  Filename  :=  _ 

”C:\Documents  and  Settings\SEEDLab\NorfolkMOE\Norfolk_'’  &  Name  & 
ABG.AgentAlpha, Beta, Gamma”  _ 

,  Origin:=437,  StartRow:=l,  DataType:=xlDelimited,  TextQualifier:=  _ 
xlDoubleQuote,  ConsecutiveDelimiter:=False,  Tab:=False,  Semicolon:=False  _ 

,  Comma:=True,  Space:=False,  Other:=False,  FieldInfo:=Array(Array(l,  1),  _ 

Array(2,  1),  Array(3,  1),  Array(4,  1),  Array(5,  1),  Array(6,  1),  Array(7,  1),  Array(8,  1),  _ 

Array(9,  1),  Array(10,  1),  Array(ll,  1),  Array(12,  1),  Array(13,  1),  Array(14,  1),  Array(15  _ 

,  1),  Array(16,  1),  Array(17,  1),  Array(18,  1),  Array(19,  1),  Array(20,  1),  Array(21,  1),  _ 

Array(22,  1),  Array(23,  1),  Array(24,  1),  Array(25,  1),  Array(26,  1),  Array(27,  1),  Array(  _ 

28,  1),  Array(29,  1),  Array(30,  1),  Array(31,  1),  Array(32,  1),  Array(33,  1),  Array(34,  1),  _ 

Array(35,  1),  Array(36,  1),  Array(37,  1),  Array(38,  1),  Array(39,  1),  Array(40,  1),  Array(  _ 

41,  1),  Array(42,  1),  Array(43,  1),  Array(44,  1),  Array(45,  1),  Array(46,  1),  Array(47,  1),  _ 

Array(48,  1),  Array(49,  1),  Array(50,  1),  Array(51,  1),  Array(52,  1),  Array(53,  1),  Array(_ 

54,  1),  Array(55,  1),  Array(56,  1),  Array(57,  1),  Array(58,  1),  Array(59,  1),  Array(60,  1),  _ 

Array(61,  1),  Array(62,  1),  Array(63,  1),  Array(64,  1),  Array(65,  1),  Array(66,  1),  Array(  _ 

67,  1),  Array(68,  1),  Array(69,  1)),  TrailingMinusNumbers:=True 
Range(”B2:CC4”).Select 
Selection.Copy 

'  change  name  of  active  window 
Windows(”DP”  &  dp  &  ’’Output”). Activate 

Selection.PasteSpecial  Paste :=xlPaste Values,  Operation:=xlNone,  SkipBlanks  _ 

:=False,  Transpose:=False 
’  change  name  of  active  window 

Windows(”Norfolk_”  &  Name  &  ”_Agent  ABG.AgentAlpha,Beta,Gamma”).Activate 

ActiveWindow.  Close 

’  change  i  =  xOl  To  (x+l)00 

For  i  =  ((dp  *  100)  -  98)  To  (dp  *  100) 

Workbooks. OpenText  Filename :=  _ 

”C:\Documents  and  Settings\SEEDLab\NorfolkMOE\Norfolk_”  &  i  &  ’’Agent 
ABG.AgentAlpha,Beta,Gamma”  _ 

,  Origin:=437,  StartRow:=l,  DataType:=xlDelimited,  TextQualifier:=  _ 

_ xlDoubleQuote,  ConsecutiveDelimiter:=False,  Tab:=False,  Semicolon:=False  _ _ 
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,  Comma :=Tme,  Space:=False,  Other:=False,  FieldInfo:=Array(Array(l,  1),  _ 

Array(2,  1),  Array(3,  1),  Array(4,  1),  Array(5,  1),  Array(6,  1),  Array(7,  1),  Array(8,  1),  _ 
Array(9,  1),  Array(10,  1),  Array(ll,  1),  Array(12,  1),  Array(13,  1),  Array(14,  1),  Array(15 
,  1),  Array(16,  1),  Array(17,  1),  Array(18,  1),  Array(19,  1),  Array(20,  1),  Array(21,  1),  _ 
Array(22,  1),  Array(23,  1),  Array(24,  1),  Array(25,  1),  Array(26,  1),  Array(27,  1),  Array(  _ 
28,  1),  Array(29,  1),  Array(30,  1),  Array(31,  1),  Array(32,  1),  Array(33,  1),  Array(34,  1),  ^ 
Array(35,  1),  Array(36,  1),  Array(37,  1),  Array(38,  1),  Array(39,  1),  Array(40,  1),  Array(  _ 
41,  1),  Array(42,  1),  Array(43,  1),  Array(44,  1),  Array(45,  1),  Array(46,  1),  Array(47,  1),  _ 
Array(48,  1),  Array(49,  1),  Array(50,  1),  Array(51,  1),  Array(52,  1),  Array(53,  1),  Array( 
54,  1),  Array(55,  1),  Array(56,  1),  Array(57,  1),  Array(58,  1),  Array(59,  1),  Array(60,  1),  _ 
Array(61,  1),  Array(62,  1^  Array(63,  1^  Array(64,  1^  Array(65,  1),  Array(66,  1^  Array( 
67,  1),  Array(68,  1)),  TrailingMinusNumbers:=Tme 
Range(”B4:CC4”).Select 
Selection.Copy 

'  change  name  of  active  window 
WindowsC’DP”  &  dp  &  ”Output.xls'’).Activate 
'  change  (i-x)  to  design  point  start  -1 
Cells((i  -  ((dp  *  100)  -  100)  +  2),  l).Select 

Selection.PasteSpecial  Paste :=xlPaste Values,  Operation :=xlNone,  SkipBlanks  _ 

:=False,  Transpose:=False 

Windows("Norfolk_”  &  i  &  ”_Agent  ABG.AgentAlpha,Beta,Gamma'’).Activate 
Active  W  indow.  Close 
Next  i 

WindowsC’DP”  &  dp  &  ’’Output”). Activate 
ActiveW  orkbook.  save 
ActiveWindow.  Close 
Next  dp 
End  Sub 


TruncateO:  When  it  is  time  to  ereate  the  formulas  needed  to  generate  the  MOEs 
of  interest  from  the  raw  data,  a  problem  arises  when  using  eonditional  statements  for  an 
agent  elass  type.  Eaeh  agent  has  its  elass  name  listed  followed  by  a  number  indieating 
the  speeifie  agent  number  for  that  run.  An  example  is  Hostile2Target  (5),  where  the  data 
represents  the  5^^  agent  for  that  run,  whieh  is  of  elass  type  Hostile2.  In  order  to  perform 
eonditional  ealeulations  for  all  agents  of  a  elass  type,  the  speeifie  agent  identifieation 
number  needs  to  be  truneated.  The  following  program  truneates  the  agent  identifieation 
from  the  header  line  of  the  design  point  file.  This  is  done  by  defining  the  header  eolumn 
as  delimited  data  where  the  delimiter  is  the  first  open  parenthesis.  Therefore,  the  newly 
formed  header  will  inelude  all  previous  information,  regardless  of  string  length,  minus  all 
text  following  the  first  parenthesis.  This  proeess  is  repeated  for  all  257  design  point  files. 
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Sub  TmncateO 

f 

'  Truncate  Macro 

'  Macro  recorded  8/27/2006  by  SEEDLab 


For  dp  =  1  To  257 
Workbooks. Open  Filename :=  _ 

'’C:\Documents  and  Settings\SEEDLab\NorfolkMOE\DPOutput\DP'’  &  dp  &  ’’Output.xls” 

Rows('T:l'’). Select 

Selection.Copy 

Sheets(”Sheet2”).Select 

Selection.PasteSpecial  Paste :=xlPaste All,  Operation :=xlNone,  SkipBlanks:=  _ 

False,  Transpose:=True 
Application.CutCopyMode  =  False 

Selection. TextToColumns  Destination:=Range('’Al'’),  DataType:=xlDelimited,  _ 
TextQualifier:=xlDoubleQuote,  ConsecutiveDelimiter:=False,  Tab:=False,  _ 
Semicolon:=False,  Comma:=False,  Space:=False,  Other:=True,  OtherChar_ 

FieldInfo:=Array(Array(l,  1),  Array(2,  1)),  TrailingMinusNumbers:=True 
Selection.Copy 
SheetsC’  Sheet  1  ”) .  Select 
Range(”  A 1  ”)  .Select 

Selection.PasteSpecial  Paste:=xlPasteAll,  Operation :=xlNone,  SkipBlanks:=  _ 

False,  Transpose:=True 
ActiveWorkbook.Save 
ActiveW  orkbook.  Close 
Next  dp 

End  Sub 


InsertColQ:  First,  a  clone  of  the  first  design  point  file  is  created  and  it  is  renamed 
“MasterOutput.”  This  file  is  used  to  create  all  the  MOE  formulas/columns  of  interest. 
Once  the  desired  MOE  columns  are  created,  the  following  code  inserts  new  columns  in 
each  design  point  file  that  correspond  to  the  number  of  MOE  formula  columns.  These 
columns  are  added  the  beginning  of  each  design  point  file,  so  that  when  they  are  later 
merged,  the  same  columns  will  be  copied  from  each  design  point  file,  regardless  of  how 
many  columns  each  file  contains. 

Sub  InsertcolQ 

f 

'  Insertcol  Macro 

'  Macro  recorded  8/27/2006  by  SEEDFab 

t 

t 

For  dp  =  1  To  257 
Workbooks.Open  Filename:=  _ 

"C:\Documents  and  Settings\SEEDLab\NorfolkMOE\DPOutput\DP"  &  dp  &  "Output.xls" _ 
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For  i  =  1  To  33 

ColumnsC’A:  A”). Select 
Selection.Insert  Shift:=xlToRight 
Next  i 

ActiveWorkbook.Save 
ActiveWorkbook.Close 
Next  dp 
End  Sub 


FormulasQ:  This  program  copies  the  formulas  created  in  the  MOE  columns  of 
the  MasterOutput  file  and  places  them  in  the  newly  formed  MOE  columns  of  each  design 
point  file.  The  result  is  automatically  calculated  MOEs  for  each  design  point  file. 

Sub  FormulasO 

f 

'  Formulas  Macro 

’  Macro  recorded  8/27/2006  by  SEEDLab 


For  dp  =  1  To  257 

RangeC’Al:AG102”).Select 

Selection.Copy 

Workbooks. Open  Filename :=  _ 

”C:\Documents  and  Settings\SEEDLab\NorfolkMOE\DPOutput\DP"  &  dp  &  ” Output.xls ” 
Range(”Al”).Select 

Selection.PasteSpecial  Paste :=xlPasteFormulas,  Operation :=xlNone,  _ 

SkipBlanks:=False,  Transpose:=False 
Application.  CutCopyMode  =  False 
Calculate 

ActiveWorkbook.Save 
Active  W  orkbook.  Close 
WindowsfOutputMaster").  Activate 
Next  dp 
End  Sub 


MergeO:  This  program  copies  all  MOE  columns  from  each  design  point  file  and 
pastes  them  alongside  the  corresponding  input  data  created  using  the  DPIterate  and 
RowCopy  programs.  The  result  is  a  single  file  with  25,700  rows  of  data.  The  first  33 
columns  represent  the  input  parameters  while  the  remaining  columns  represent  the  MOE 
data,  or  output  values.  The  file  is  now  ready  for  analysis  using  one’s  favorite  statistical 
software  package. 
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Sub  merge() 

For  dp  =  1  To  257 
Workbooks. Open  Filename :=  _ 

”C:\Documents  and  Settings\SEEDLab\NorfolkMOE\DPOutput\DP”  &  dp  &  ” Output.xls ” 
RangeC’B3:AG102”).Select 
Selection.Copy 

W  indowsC’  S  ingleOutput” ) .  activate 
Cells(((dp  *  100)  -  100)  +  2,  35).Select 

Selection.PasteSpecial  Paste:=xlPasteValues,  Operation :=xlNone,  SkipBlanks  _ 

:=False,  Transpose:=False 
WindowsC’DP”  &  dp  &  ’’Output”). activate 
ActiveWorkbook.Save 
Active  W  orkbook.  Close 
W  indows(”  S  ingleOutput” ) .  activate 

Next  dp 

End  Sub 
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APPENDIX  D.  MILITARY  OPERATIONS  RESEARCH  SOCIETY 
(MORS)  TISDALE  COMPETITION  PRESENTATION 


The  following  slides  were  presented  to  a  eommittee  of  faculty  at  the  Naval 
Postgraduate  School  for  the  Military  Operations  Research  Society  (MORS)  Stephen  A. 
Tisdale  Graduate  Research  Award.  This  award  recognizes  high-quality  research  of 
immediate  or  near-term  value  to  the  defense  of  the  United  States  and  its  allies.  This 
author  was  one  of  five  finalists  selected  for  the  competition  based  on  the  potential  impact 
of  this  thesis  work  on  the  Department  of  Defense. 


69 


What’s  the  Problem? 


Cvtm  ^C!sie«r«rit 


Can  We; 

^potential  threats? 

of  approaching 

small  vessels? 

from  crossing  the 
exclusion  zone? 


Do  We: 

-  Have  the  and 
place  to  answer  these  questions? 


P.r-^ 


in  ^  r 

ft  0 


■li 


idST' 

3r‘ 


;•■ -Tid 

‘f  ■,  “ 


J  - 


1 


Optical  Dazzler  Acoustic  Haiiing  Device 


70 


Research  Questions 

Primary  research  question: 

•  What  non-lethal  capabilities  are  required  for  maritime 
force  protection  missions? 

Supporting  questions: 

-  Is  Multi-Agent  Simulation  (MAS)  an  appropriate  tool? 

-  When  are  non-lethal  capabilities  tactically  appropriate? 

-  What  are  the  geographical  effects? 

-  How  to  identify  threats  from  non-threats? 


Methodology 

•  Agent  Based  Modeling 

•  “Data  Farming” 

-  Efficient  Design  of  Experiment 

-  Distributed  Simulation 

•  Data  Analysis 


The  Agent  Based  Model 
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Efficient  Design  of 
Experiment 


Gridded 

NOLH 

#  of  Factors 

29 

29 

#  of  Levels/Factor 

2 

~10  (up  to  40!) 

#  of  Design  Points 

536870912 

257 

#  of  Runs/DP 
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100 

Total  CPU  Time 

1021  years 

18  days 
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Data  Analysis 
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•  Uses  Multiple  Analysis  Techniques 

-  Multiple  Regression 

-  Classification  Trees 

-  Data  Visualization 

•  Contour  Plots 

•  3-D  Surface  Plots 
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THSpd>=7  j 

TTHSpd<7 

Count  165 

Count  37 

Mean  0.026102 

Mean  0.2424955 

Std  Dev  0.0851433 

Std  Dev  0.3080686 
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LDRedRad>=7 


Count  25 

Mean  0.5774572 
Std  Dev  0.3177766 


||LDRedRad<7  || 

Count  30 

Mean  0.8570788 

Std  Dev  0.1980319 

[WMRedRad>=19  HI 

Count  14 

Mean  0.0192381 
Std  Dev  0.0310368 


|rWMRedRad<19  "j 

Count  23 

Mean  0.3783913 
Std  Dev  0.3220342 


1  AHDFirRate<0.39  ] 

l>HDFirRate>=0.39  ~|| 

Count  15 

Mean  0.371916 

Std  Dev  0.2127179 

Count  10 

Mean  0.885769 

Std  Dev  0.1512221 

AHDFirRate<0.34 

Count  12 

Mean  0.2098611 
Std  Dev  0.234176 


AHDFirRate>=0.34 

Count  11 

Mean  0.5622424 
Std  Dev  0.310752 
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ROE/Training  vs. 
Requirements 


MOE:  %  Hostile  Targets  Identified  (and  engaged) 
outside  of  the  NPEZ 
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Count  25 
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Count  14 
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Std  Dev  0.0310368 
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Count  15 
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Conclusions 


D 

Faring 


•  Hostile  Targets: 

-  Tactical:  Tactically  appropriate  against  targets  w/  speed  <  14 
knots. 

-  Requirements:  AMD  significantly  more  effective  when  used  as 
audio  warning  only  and  not  message  delivery. 

•  Neutral  Targets: 

-  Tactical: 

•  Geography  plays  an  important  role  in  deterring  neutral  targets. 

•  The  tactic  modeled  is  significantly  less  effective  when  environmental 
conditions  cause  performance  degradation  of  AHD. 

-  Requirements:  The  first  response  weapon  has  the  most  effect. 

•  Loitering  Targets: 

-  Significantly  effective  in  identifying  threats  from  non-threats 

-  When  used  alone,  counter-personnel  non-lethal  capabilities  fail 
to  deter  loitering  targets  who  attack  when  within  close  proximity. 
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The  Impact 


Outlines  Capability  Requirements 

-  Compliments  SME  in  defining  requirements 

-  First  to  evaluate  capabilities  as  a  system 


•  Validates  Tactics 

-  Explores  integration  with  current  tactics  early  in  the 
process 

-  Assesses  tactics  in  varying  geo-political  situations 


•  Revolutionizes  Vulnerability  Assessments 

-  Evaluates  nearly  any  possible  enemy  COA 

-  Exploits  vulnerable  areas  of  interest 


Who  Cares? 


•  Naval  Postgraduate  School 

•  Joint  Non-Lethal  Weapons  Directorate 

•  OPNAVN757 

•  Mobile  Security  Forces,  Norfolk 

•  OPNAVN81 

•  Office  of  Naval  Research 

•  Project  Albert  -  Referentia  (MCWL) 

•  Joint  Test  and  Evaluation  Methodology  (JTEM) 
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Now  this  is  not  the  end.  it  is  not  even  the 
beginning  of  the  end.  But  it  is,  perhaps,  the  end 
of  the  beginning. 

-  Winston  Churchill 
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